Arbitra

An A-level Computer Science project by Samuel Newman

ArbitraIntroductionProject AimsLanguages and softwareNamesUsageAnalysisResearchA brief introduction to CryptocurrenciesMore on walletsObjectivesBasic ProtocolPrototyping Core FunctionsHashing and BlocksElliptic Curve Digital Signature AlgorithmDisclaimerElliptic CurvesPoint AdditionFinding the line between a and bFinding the third pointPython ImplementationPoint at InfinityPoint MultiplicationThe PatternThe SolutionFinite FieldsImplementing Modular Inversemod pSide note regarding notationGraphs and mod pThings we know about the point at infinityObjectifyingBack to CryptographyPicking a curveCurve CharacteristicsCreating a signatureVerifying a signatureVerifying the programConclusionNetworkingApplication planningConceptConcept 1Concept 2Concept 3PagesKeyDocumented DesignHow the network system worksA closer look at the ProtocolData typesHeaderMessage typesTransactionBlockPingNode RequestLatest Block Hash RequestChain RequestReply typesPing replyChainBlock hashNodeReceivedError MessageNetwork diagramsSimulation 1: From S1Simulation 2: From S1User Interface DesignConcept 1RedesignSource codeHTML - index.htmlCSS - style.cssListsCreating the Electron appRemoving the FrameChanging pagesHighlighting Menu LinksThe crypto ModuleNetworkingServer ExampleClient ExampleData FilesFile PlanningTechnical SolutionConverting Python CodeECDSAsha256()onCurve()invMod()addPoints()multiPoints()createKeys()signMsg()verifyMsg()TestingSimplifying FunctionsHashingFilestore()Removing arraysTestingget()getAll()storeAll()append()Message Sending/Receivingconnect()setInterval()Message Parsing/ProcessingparseMsg()Parsing Different Message Typespg()TestingRestructuring parseMsg()tx()transaction()bk()hr()nr()cr()Sending MessagesParsing repliesParsing different reply typescn()nd()bh()ok()er()Exporting functionsBlockchaingetBlock()BalancescalcBalances()getTopBlock()mainChain()getChain()checkBalance()addBlock()PagesTransactionsWalletsCreating WalletsTestingCreate TransactionView recentBlockchainMiningMulti-threading alternativesMining pageMining ScriptTestingViewingSettingsNetwork SettingsManual PingTarget ConnectionsAdvertiseClear connectionsApplication SettingsFinal TouchesMoving CSSHeight counterIcon and Splash ScreenOverview pagenpm start scriptTestingTest 1 - Application fresh startMethodExpected OutcomesTest ResultProofTest 2 - Creating a walletMethodExpected OutcomesTest ResultProofTest 3 - Pinging a clientMethodExpected OutcomesTest ResultProofTest 4 - Automatic reconnectingMethodExpected OutcomesTest ResultProofTest 5 - Connected to backupMethodExpected OutcomesTest ResultProofTest 6 - Mine a blockMethodExpected OutcomesTest ResultProofIssuesFixesTest 7 - Make a transactionMethodExpected OutcomeTest ResultProofTest 8 - Add a transaction to blockchainMethodExpected OutcomesTest ResultProofTest 9 - Save wallets.jsonMethodExpected OutcomesTest ResultProofTest 10 - Invalid transactionMethodExpected OutcomesTest ResultProofEvaluationInitial ObjectivesPersonal ThoughtsCodeappstaticpagesjspages
















Introduction

Arbitra is a cryptocurrency – a way of performing transactions that are guaranteed by the rules of mathematics rather than a central bank or some other 3rd party. A blockchain shared amongst a decentralized peer-to-peer network is used to verify transactions.

Project Aims

  1. Describe in detail how the cryptocurrency would work. There needs to be detailed documentation on how a client application would work, and the protocols with which the network would work. It must be in sufficient detail such that a fully-compliant client could be built by a third party using only this document as reference.
  2. Implement a client application with which one would interact with the network. This includes: -- Hashing -- Public-key cryptograph -- Blockchain technology -- Peer-to-peer networking -- Electron desktop application
  3. Use the client on a small network of computers, creating a network and thus creating a cryptocurrency.

Languages and software

The initial plan is to use the Electron framework. This means that the frontend will be using HTML/CSS, and the backend will use Javascript with a bit of Node.js, the Javascript runtime that is usually used for web server backends. This is helpful because the nature of cryptocurrencies involves a lot of peer-to-peer communication, which Node should excel at.

https://electron.atom.io

https://nodejs.org/en/

The reason I chose Javascript and Electron is that I have a lot of experience with web development, but next to no experience in the GUI systems of any other of the languages I know. Given the time constraints, having to learn a new language could be risky because of how long it would take. Electron will allow me to work with HTML and CSS, which I am very comfortable with, and using Javascript has turned out to have other benefits. With access to the Node Package Manager (npm), I should have no problem finding a cryptography package or anything else I might need.

Something to consider is that Bitcoin and Litecoin were made in C++ and Ethereum was made in Golang. However, since I have no experience in either of those languages I thought it would be safer to stick with what I am comfortable with. If performance with Javascript turns out to be too much of an issue I could potentially write the performance-critical parts of the application in a lower-level language like C++ but still use Javascript for the UI/networking/other logic.


Names

The cryptocurrency is called Arbitra. However, the unit of currency is an Arbitrary Unit, or au.

The reason it is called Arbitra is because I thought it would be funny to have the unit of currency be an arbitrary unit. It also happens that using the shortened term is the same as the chemical symbol for gold, which has an amusing juxtaposition between something worthless (an arbitrary unit) and something valuable (gold).

Usage

I really like Arbitra. I have over 100 arbitrary units!

That will cost you 50au.

Analysis

Research

The Bitcoin whitepaper (https://bitcoin.org/bitcoin.pdf) was used to understand much of how a cryptocurrency works. "Ever wonder how Bitcoin (and other cryptocurrencies) actually work?" (https://www.youtube.com/watch?v=bBC-nXj3Ng4) by 1Brown3Blue was also very helpful. "Building a desktop application with Electron" (https://medium.com/developers-writing/building-a-desktop-application-with-electron-204203eeb658) was useful as reference when building the Electron app, as well as the Electron Quick Start Guide (https://electron.atom.io/docs/tutorial/quick-start/).

A brief introduction to Cryptocurrencies

A transaction, most simply, is a message that says "I want to send this person x arbitrary units". The idea of a cryptocurrency is that rather than having a physical unit that you hand over to someone, you instead have a list of all transactions ever made and determine your account balance from that. This means that everyone can be certain that no-one is forging money or faking transactions - everyone can see the list and validate it. It also removes trust out of the equation - you don't have to just hope that your bank is keeping your money safe.

To create a transaction, you must have a “wallet” that has some currency in, and use a public/private key pair to cryptographically “sign” your transaction. Public/private key pairs are a cryptographic function that allows a private key to create a message with which one can use the public key to mathematical verify that the message was created with it's paired private key. You would then submit your transaction to a node on the network. If your transaction is valid, the node will send it to all other nodes in the network who will add it to a “block” of transactions.

A block consists of all transactions submitted since the previous block, the timestamp, the hash of the block that came before, and a nonce (which I’ll get to). Having the hash of the previous block irrevocably links the block to the one that came before it, and since it is easy to check, it is impossible to change the previous block’s contents without changing all the blocks that have come after. It also ensures that they are ordered. These blocks thereby form a chain from the genesis block, hence “blockchain”.


 

This is an example of a transaction, represented in JavaScript Object Notation (JSON). A block would be a collection of these.

A blockchain is the equivalent of a bank's ledger - all transactions are recorded to keep track of everyone's balance. The difference between a ledger and a blockchain is that a ledger is kept secret by the banks, whereas the blockchain is held by anyone who wants it. This guarantees that the transactions cannot be messed with.

Anyone can download the blockchain and see if it is valid. But how do you stop someone adding a malicious yet technically valid block to the chain? This is what the nonce is for. In order to be added to the blockchain, the block is hashed using SHA-256 and if the hash meets some criteria, it is sent to all the other nodes in the network. Otherwise, the nonce is changed over and over until the right hash is found. This means that it takes a lot of work to find a valid block, and this work can be verified almost instantly.

This means that if someone wants to submit a malicious block, they will have to either get impossibly lucky, or have more computing power at their disposal than the rest of the network, which is infeasible. Furthermore, even if you do manage to validate a malicious block, nodes will always accept the longest chain, so you would have to keep adding to your malicious chain effectively forever, which gets exponentially harder.

This means, to become a node in the network, you (or your computer) would need to:

  1. Collect transactions until the next block is added to the chain.
  2. Start iterating the nonce and hashing the block, while collecting transactions for the next block. a. If you find the right hash, broadcast it to the network. b. If a completed block is sent to you, check if it is valid. If so, accept it.
  3. Start hashing the next block.

Hashing the block in order to find the correct nonce is called “mining”.

In order to incentivize mining, each block contains a transaction at the top that gives some amount of currency to whoever mines the block. This means that a miner can recoup the cost of the electricity and equipment spent mining the block. It also means that, if you had 51% of the CPU power of the network, it could well be more profitable to play by the rules and get the mining rewards rather than stealing coins.

Unlike in Bitcoin and most other cryptocurrencies, Arbitra will not reduce the mining rewards over time to cap the number of coins. This is to avoid the issue of requiring transaction fees, as this penalizes poorer users. It does mean that Arbitra will inflate over time, but unlike fiat currencies it will be completely predictable.

Something else to consider is that once a block is added it does not mean that it is there to stay. Even if a block is mined, if another block is also found at the same time (a "branch") it depends on which one ends up having the longer chain follow it. Having multiple competiting blocks is intentional, to ensure legitimacy, but this also means that the "top layer" of the blockchain is not necessarily trustworthy.

However, since it becomes exponentially harder to keep up with the main chain if a branch starts lagging behind. This should ensure that smaller branches die off quickly

A client application, which is what most ordinary people would use, does not do any mining. It simply checks the blockchain to see how much money is attributed to it, and it can sign and send transactions to the network. In this project I will make both a client and a node, but wrapped into one application for simplicity's sake.

As you can see, there is no need to trust any one central authority in the network once it starts. For as long as the cryptographic principals hold, the currency is effectively a democracy based on computing power - the blockchain is controlled by those who hold the majority of computing power, and this will almost certainly never be controlled by one party.

More on wallets

Something that must be cleared up - a wallet is not like a bank account. All that is needed for a wallet is a private key, which can be created freely. You also don't need to register your wallet with any central authority, once again ensuring the honesty and security of the network. All a wallet is is a random number. It also solves the problem of everyone seeing every transaction - it doesn't matter because your balance is split up amongst a bunch of anonymous keys.

Objectives

These objectives estimate what must happen for the project must be a success.

Basic Protocol

From the description of the cryptocurrency, we can determine the format of the most important types messages that would be sent in the Arbitra network. Most simply, the basic message types and their contents are:

When a new node joins the network, it will send a Latest Block Request to a few nodes, and will use the previous block’s hash to fill in the chain from the top. If the Latest Block Request returns a few different blocks, the system should default to the longest chain, and only use the other blocks if no other nodes recognize that block. The specific details on how that process should work needs investigation.

Prototyping Core Functions

In order to working out the specifics of how this is going to work, I decided to use Python to implement some of the functions that a real client would use. The final client will be an Electron app made with Node.js, but due to the ease of iteration in Python I think that using Python for testing purposes would be easier and simpler.

Hashing and Blocks

First off, I needed to make a simple hashing function. I created a wrapper around the SHA256 function from the hashlib library so that I could easily get the SHA256 hash of a string.

 

First, I import hashlib at the top of the program. Then, I define a function called sha256(), which takes in a variable called inputstr. sha256() creates a hashlib.sha256() object, which we can use to create a hash. We then update() the hashlib object with the string we want to hash. Notice we encode inputstr, which is because hashlib can only hash bytes, not characters, so we call encode("utf-8") on it. Finally, we return the hexdigest() of the object. This converts the object into a hex string.

This produces the following result:

 

It worked. We now can hash any string. Therefore, the next step is to prototype mining. The next program should then, while the output does not fit a criteria, repeatedly hash a random string with a nonce. To fit the criteria, the string must begin with a certain number of zeros.

 

mine(), like the other function, takes an input called inputstr. It then sets up the constant DIFFICULTY and the initial variables, nonce and fail. DIFFICULTY is the number of zeros that need to be at the beginning of the hash for it to pass. nonce is (obviously) the nonce. This iterates each time the hash does not pass. fail is a boolean that is false, and will set to true if any one of the first 4 (or whatever the difficulty is) characters of the hash is not zero.

There is then a while loop that will loop forever until it is broken. Then, inputstr and nonce are hashed using the sha256 function we made earlier, and the hash is assigned to hashed When the hash is found, a for loop is used to see if the any of the first few characters of the hash are not zero. If so, fail is set to True so that when the loop is over it fails the test if any one of the first few characters are zero.

If the test did not fail, it prints out the input, the final nonce and the final hash, then breaks the while loop, ending the function. If it were a proper function, it would return these values instead of printing them If it did fail, it iterates the nonce and resets fail. It also prints the nonce so that you can see the progress.


 

It found a hash that begins with 4 zeros in only 54395 iterations. This took quite a long time (9 minutes 30 seconds). This is way too long - the target time is (currently) 5 minutes. However, this was only one laptop. With a network of computers around the world checking random nonces and a different difficulty, this can be achieved. In fact, as the overall computing power of the network increases, the difficulty will need to increase with it. To confirm that it works, I changed the difficulty to 2 and made it so that it also prints the hash as well as the nonce when it fails.

 

As you can see, with the difficulty reduced to 2 it only took 346 iterations, which took about 5 seconds. In this way, we can dynamically set the difficulty so that as the computing power of the network increases we can make sure that the time to mine each block stays about the same. If we want more granular control of the difficulty, we could make it so that it requires, for example, 2 zeros and 3 numbers less than 5.

At this early stage, statements like these are guesswork, but I think that there will be a target time (5 minutes) and if a block is significantly earlier than that the target time the number of zeros increases by one and vice versa if it takes too long.

Elliptic Curve Digital Signature Algorithm

To sign messages in Arbitra, we are going to use the Elliptic Curve Digital Signature Algorithm, or ECDSA. As previously explained, Digital Signatures are a way of verifying that a message was sent from someone, by including a private key that can only have been generated by the private key, which only the sender should know. We can use a mathematical concept called Elliptic Curves to sign the messages in Arbitra.

Rather than using the Node.js crypto module's implementation, I decided to implement it myself, so I understand what's happening behind the scenes. I found a paper called Implementation of Elliptic Curve Digital Signature Algorithm, which provides some insight into the mathematics behind ECDSA, as well as an article called Understanding How ECDSA Protects Your Data. The best article I found was called Elliptic Curve Cryptography: a gentle introduction, as it covers the mathematics in depth enough to implement, but not assuming prior knowledge as the majority of other articles did, as well as providing Python examples of some algorithms.

https://pdfs.semanticscholar.org/c06a/d6512775be1076e4abd43e3f2928729da776.pdf

http://www.instructables.com/id/Understanding-how-ECDSA-protects-your-data/

http://andrea.corbellini.name/2015/05/17/elliptic-curve-cryptography-a-gentle-introduction/

Disclaimer

The reason I decided to implement ECDSA myself is that I could not figure out how to do it in the crypto module, and when I realised how to do it by that point I'd already done way too much research to back down. Still, it has proved a valuable learning experience.

Elliptic Curves

Elliptic curves are curves of the form .

some curves from wikimedia

But how does this relate to cryptography?

From http://wstein.org/edu/2007/spring/ent/ent-html/node89.html:

Suppose is an elliptic curve over and . Given a multiple of , the elliptic curve discrete logarithm problem is to find such that .

This means that it's really hard to find when you only have and - so hard, that we can use this problem to prove mathematically that has been generated by . This is how we sign messages in Arbitra - it shows that the message has to have been signed by the person who has the secret key.

However, before we start that we need to understand the operations we can perform on points on the curve, which is different to normal arithmetic. There are some operations we can perform:

Point Addition

Point addition is the process of adding two points together to find a third. This is best shown by this interactive Desmos graph: https://www.desmos.com/calculator/ialhd71we3

desmos point addition

To add two points, you draw a straight line between them, and then find the negative of the third root when this line meets the curve.

This is very simple to calculate graphically, and relatively simple to calculate with pen and paper. However, to create a function we will need to create a formula using and .

It would be easier to just have to input and and for the function to calculate the corresponding values. Using the curve , we can find them using:

Finding the line between a and b

To find the equation of the line between the points, we need to find the gradient of the line, which is given by:

The equation of the line is of the form , and since we have values for , , and , we can rearrange to find :

This is the -intercept of the straight line.

Finding the third point

is given by:

We need to find . We can use:

The point we are trying to find is therefore given by:

Python Implementation

I wrote this into a Python function, where the curve is

 

From the interactive Desmos graph, inputting values of 1 and 3 gave .

 

The function works. It doesn't handle edge cases yet, but we'll cover that later.

Point Doubling

The second operation we can perform is point doubling. From this graph:

point doubling from instructables.com

It looks as if point is the negative of the second intercept of the tangent from point . This makes sense, as , and doing point edition with the same point will result in the tangent of that point. Unfortunately, since we find the gradient of the line using , if that would divide by zero. Therefore we need to differentiate using the chain rule:

We can now implement this in Python - I just used the same addPoints() function for simplicity. Also more comments.

 
 

We can verify this result using this tool, from Elliptic Curve Cryptography: a gentle introduction, since it takes this into account while the other tool doesn't.

https://cdn.rawgit.com/andreacorbellini/ecc/920b29a/interactive/reals-add.html?px=1&py=2&qx=1&qy=2

Therefore this function works. Before we finish, I wanted to make sure we could easily change the curve, by changing it to . This means that we have to recalculate some of the maths, most notably the tangent equation:

I also noticed we could simply the tangent equation given we know already. All the changes that were necessary to the python function were changing the differential and changing how it finds y1 and y2.

 

When a = 0 and b = 7 it gives the same results:

 

With a = -7 and b = 10, it gives:

 

Which can be verified with the tool:

https://cdn.rawgit.com/andreacorbellini/ecc/920b29a/interactive/reals-add.html?px=1&py=2&qx=1&qy=2

Point at Infinity

What happens if the line doesn't intersect with the curve

If the line does not intersect, we say that it intersects the point at infinity, . This only happens if you try to double where . However, this is not critically important at this phase, although we will have to add this in as an exception when creating the real function later on, as it will break the function (which is not good).

Point Multiplication

Now that we can do the basic function, we need to be able to multiply points, as we need to get to the stage where we can calculate . The obvious way would be to double , and then calculate for number of times. However, as Elliptic Curve Cryptography: a gentle introduction points out, this equation has an efficiency of , which is not particularly fast. However, they suggest using the double and add method to multiply points.

http://andrea.corbellini.name/2015/05/17/elliptic-curve-cryptography-a-gentle-introduction/#scalar-multiplication

In summary, if you repeatedly double , you would get the pattern:

And we can can represent a decimal number in the following form:

Therefore:

Since we can find using point doubling, and since we can represent a large number very easily using binary, we can massively cut down on the calculations needed to multiply a point. In our example, five doublings and one addition is needed to find and in the article, they show that only seven doublings and four additions are needed to find . This brings the complexity down to , which is much faster for larger numbers.

algorithmic complexity graph

We can implement this in python (note that this is without looking at the given example code):

 

The reason that it takes the first three digits off of the binary string is that they firstly are not of fixed length, so always start with 1, and secondly begin with 0b. Knowing this, we can remove the first 3 digits and set total to start with the value .

This gives:

 

We can verify this with this calculator:

https://cdn.rawgit.com/andreacorbellini/ecc/920b29a/interactive/reals-mul.html

Unfortunately it does not work, and it turns out the reason is that I made a logical error in the code. Even though the first digit of the binary sequence is always , because that is the most significant bit it is actually the last number we want. The least significant bit, , can be or , and so the corrected code looks like this (with total now initialising to and the loop rearranged):

 

Strangely enough, this doesn't work for some numbers, but still does for others - looping though 0 to 24 produces the correct answers for 1, 2, 3, 4, 8, 9, 10, 11, 16, 17, 18, 19, and 24 (with the caveat that the y value is always positive because of the square root), but incorrect answers for the rest. Why is that? For 0 it is because there is starts at zero when it should start at infinity - that will be fixed. But the rest are strange, especially considering that it starts getting numbers incorrect, but then starts being correct again, which should not be possible considering each number relies on the previous one.

On top of this, it forms a sequence of 4 correct, 3 wrong, 4 correct, 4 wrong, which is very odd.

I decided to map out 1 to 10, with their binary equivalent, the answer the function gave, and the answer the calculator gave.

DecimalBinaryFunctionCalculatorCorrect?
1155Yes
2100.65340909090909170.6534Yes
311-1.562727156221965-1.56275Yes
4100-1.250473444973121-1.25046Yes
5101-1.56272715622196531.07934No
61100.65340909090909187.3407No
71115.000000000000002169.37407No
810003.59150535055093733.59139Yes
910010.243722943837772380.24367Yes
101010-1.7888421516340784-1.78888Yes

Some observations:

The Pattern

I noticed that when it started to go wrong, it was after the previous x value was negative, which I confirmed using the calculator. What could cause this? Of course, the usual answer is something to do with square roots. If we look at the addPoints() function:

 

There it is - calculating y1 and y2 uses math.sqrt(). This effectively means that it only adds points above the line . We need to work around the square root in order for the algorithm to work in all cases.

The Solution

The solution is probably to not calculate the y values in this function. Instead both the x value and the y value should be passed to the function. The best way to do this is to group points into a tuple like so: point = (4,6). Then when we want to get those variables back:

 

Alternatively we could use a list or even an object, but this is probably the simplest way.

Since I will soon be rewriting this function, I won't make changes here. As shown previously, the algorithm gets back on course after encountering a negative x value, so the multiPoint() function itself seems to work pretty well. All that is left to do is to add exceptions for multiplying by , which I will do later on.

At this point, we can now find . However, notice that we have no "point subtraction" or "point division". What if we wanted to find (the private key) from and ? That is the logarithm problem, and the point of doing all this is to make that as hard as possible - the harder it is, the more secure the algorithm is. The next step is to make it even more difficult to find .

Finite Fields

The way that ECDSA makes it harder is by using finite fields, which limits the number of elements we have. This makes the curve into a finite number of discrete points, and should make it much harder to find . The problem, as stated at the beginning of the ECDSA section, is the discrete logarithm problem.

To restrict to a finite field, we use modular arithmetic.

In summary, the modulus operation finds the remainder when dividing one number by another. There are five different operations we can do:

The last one is called the modular inverse, and is the most important. It is the equivalent of a modular division, because multiplying by an inverse number is the same as dividing by a non-inverse number.

Modular addition, subtraction, multiplication and exponentiation can all be calculated using Python's modulus operator %. We can also find the quotient without the remainder using //.

 

However, there is no built-in function to find the inverse mod of a number, so we need to implement our own algorithm to find it. The naïve approach would be to iterate through possible values of (also known as ).

 

Unfortunately that runs slowly at . We need a more efficient algorithm, and for that we need to implement the Extended Euclidean Algorithm, which has a complexity of . It takes in two numbers and , and returns and that satisfy the equation:

Where is the greatest common divisor of the two numbers.

Implementing Modular Inverse

We can calculate the easily using recursion:

 

For example:

 

However, the algorithm itself works differently.

Implementing the pseudocode found on Wikipedia:

https://en.wikipedia.org/wiki/Extended_Euclidean_algorithm#Pseudocode

 

Since Wikipedia was kind enough to provide a trace table of the algorithm, we can confirm that it works. For the input , , we expect the values , , .

 

We can further confirm that the is 2 using the gcd() function.

 

Now for the actual inverse modulus function. Something to mention at this point is that must be a prime number. This is because must be equal to so that it can be used in the modular inverse function - in fact, only has an inverse if .

 

Whilst this seems too simple to be correct, as far as I can tell that is all that is needed, since if is the inverse of :

and we know that this equation is equal to if is prime:

therefore if we find the of all values:

and since and :

This is the same form as the modular inverse. Now all we need to do is find so that we can then later use it in the form (or similar).

Now we have this algorithm down, it is time to explore the implications of .

mod p

If you were to find the modulus of all real numbers - - you would be splitting up into groups.

Here is :

mod 5

Image from Khan Academy: https://www.khanacademy.org/computing/computer-science/cryptography/modarithmetic/a/congruence-modulo

When we do that, it is known as a Finite Field, and is represented as . It contains all integers from to . In the above example, that is the finite field , and contains , , , , . Whilst this doesn't seem too useful, we can still do maths on it - except instead of normal arithmetic, we use the modular arithmetic. For example, if we wanted to do in , we would have to do , which is .

Side note regarding notation

Whilst previously I had been using , I will now be switching to , because it indicates that it's all modulus , which will be the case from now on.

Graphs and mod p

The most important thing about curves in is that instead of being a continuous line, they become a bunch of discrete points. The bigger that is, the more points there are. The axis also only goes from to (therefore "finite field").

Something else important to understand about modulus is how it "wraps around". If we were to find , it doesn't matter that as it just removes over and over until the resulting number is less than , which is the remainder. That is why it is know as "clock arithmetic", because if you wanted to find the time 50 hours after 2pm, you would go around the clock until the resulting number is less than 24. We can therefore represent this problem as , which is , and therefore 4am.

This means that a plane in the axis in the finite field would only go from to , but when a line in the graph reaches the edge of the plane it wraps around to the other side.

This means that we can still add and multiply points, as visually demonstrated in this calculator:

https://cdn.rawgit.com/andreacorbellini/ecc/920b29a/interactive/modk-add.html

However, we now need to update our point adding function. Instead of adding points on the curve , it now needs add points on the curve . This basically includes adding %p to the end of all our functions, but we need to make some deeper changes (for example to account for the point at infinity)

Before we start, two things:

  1. is a prime number. This is because when isn't prime, some points can often overlap, and we also can't find the modular inverse.
  2. is very, very big when doing cryptography. In the secp256k1 curve, for example, .

can just be an argument passed to the function in this case.

Since we will need to remake most of the functions that we've made so far, I decided to take this opportunity to clean up the code a bit. To start, I made a elliptic_curve class so that we can define , and once, and then use ec.p (for example) if we wanted .

 

In this example, we have made the curve .

Next, we need to think about how the addPoints() function works. First of all, we need to take into account the changes mentioned in the point multiplication section - passing both co-ordinates of the points.

 

Second of all, we need to make sure that the points are actually on the curve - previously, we dealt with that by only requiring one value. Now, we have to make sure the inputs make sense. To do this, we will make a new function that makes sure that by raising a ValueError if it doesn't:

 

Now addPoints() looks like this:

 

Now we need to account for the . This is simple enough.

 

However, we need to find the gradient, and there we have a problem.

Both these equations have a division in them, and we can't do that!

We need to rearrange them like so:

Now we can use our inverse modulus algorithm.

 

Finally, we can the resultant point.

 

This should work in most cases, but we're missing something - the point at infinity.

Things we know about the point at infinity

The most important point here is - that means we can just return P1 if P2 == O, and vice versa. It also means that if x1 == x2 but y1 != y2, the third point is .

The thing to notice in all these points is that none of these require to be an actual number, as if it is passed to the function, it returns something before the maths starts. Therefore, I have decided for to be the string "infinity", so it definitely cannot be confused, and will throw a TypeError if I make a mistake and it manages to get through to the maths.

With this in mind, I created the final function:

 

I tested it out with the points and , which I knew were on the curve thanks to the calculator:

 

We can verify it with this calculator:

https://cdn.rawgit.com/andreacorbellini/ecc/920b29a/interactive/modk-add.html

adding points in Fp

It worked. It should now be simple to convert the point multiplication function.

 

multiPoints() is now much simpler, because of two things:

  1. Now that we properly support , it starts total as which will give the correct answer, rather than manually setting total to the correct answer after the first run.
  2. Passing points rather than values eliminates the problem we were having with negative values.

Now for testing:

 

Which we can verify with the calculator:

https://cdn.rawgit.com/andreacorbellini/ecc/920b29a/interactive/modk-mul.html

It worked.This is all of the functions that we need to actually create signatures.

Objectifying

I decided to move all the functions into one class (elliptic_curve()), to make it simpler.

Here is the class in it's entirety:

 

We don't even need to import math anymore.

All we need to do to interact with this class is create a curve:

 

Then call the functions from that object:

 

Back to Cryptography

Now that we have all the functions we need, we can soon start actually creating and verifying signatures!

However first, we need some finally things.

Picking a curve

I found the very useful website http://safecurves.cr.yp.to that has a list of many of the elliptic curves that have been found. It also rates their security based on several different factors. Interestingly secp256k1, the curve Bitcoin uses, is not rated as safe.

However, before we pick a curve, there is something that needs to be mentioned about curves - there are different kinds. All the work so far has been for Weierstrass curves - curves with the equation that satisfy the equation .

Elliptic curves can take other forms, such as the Edwards curves of the form

Since using other curve forms would require redoing a lot of our maths, and converting from one curve type to another is very complex (I tried), I decided just to use a Weierstrass curve. Although none were deemed safe by http://safecurves.cr.yp.to/, I decided on secp256k1, which is defined as:

I chose secp256k1 for several reasons:

Curve Characteristics

The characteristics of the curve are specified in this document, from the Standards of Efficient Cryptography group:

http://www.secg.org/sec2-v2.pdf

We don't just need the the and values for the curve, we also need some other values. First of all, we need the Base Point . There is a specific value for each curve, and it's just a static number you can look up. So for secp256k1 the base point is:

But what is the base point? A base point generates a subgroup - a subsection of the points on the curve. If you find the multiples of a point, it generates a group of coordinates that eventually loops back on itself.

The order of a group is the number of points in it. The number of points in the curve as a whole is it's order, .

Therefore the subgroup order is the number of points in the subgroup created by our base point. For secp256k1:

Finally there is the subgroup cofactor, which is just . I don't believe this will come up in the calculations.

We can add these characteristics to our Python class:

 
Creating a signature

Finally, we have all the functions that we need to start signing messages.

First off, what we're signing needs to be the same bit length as , the subgroup order. Since we hash the messages using sha256, and we're using the curve secp256k1, they both have the same big length of, unsurprisingly, 256. The message we're signing is denoted as .

We also need a public key and a private key. The private key is a random integer chosen from , and the public key using the scalar multiplication function. I created a Python function to create these:

 

randrange() is from the random module, which I imported at the top of the program using from random import randint.

To create a signature, we need to follow these instructions:

  1. Choose a random integer chosen from
  2. Calculate the point
  3. Calculate where is the value of
  4. If , start again with a different value of
  5. Calculate
  6. If , start again with a different value of

From http://andrea.corbellini.name/2015/05/30/elliptic-curve-cryptography-ecdh-and-ecdsa/

The signature is then .

Implementing that in Python, as a part of the elliptic_curve() class:

 
Verifying a signature

We also can now verify a signature.

From the same source, the method to verify a secret key is:

  1. Calculate
  2. Calculate
  3. Find the point
  4. Find

If , then the signature is valid.

We can implement this in Python as well:

 
Verifying the program

We need to test the program. At first, we will sign the string "hello", and then verify it.

 

Unfortunately, this doesn't work.

 

It's a strange error that I've never had before.

It then occurred to me that this is the first time that z is operated on, and z is , the hash. The hash, if you remember the function, returns the hash as a hex string. Presumably, this is what the error is referring to. I replaced the decimal inputs with hexadecimal inputs:

 

It still didn't work. I looked up the error, and apparently it's an issue converting Python ints to a C integer. Since the change makes the code more readable, I decided to keep it regardless.

I split up the offending line to see which operation causes the error.

 
 

Even more:

 
 

So clearly, r * w is causing an overflow error. I printed both values, and it turns out the w is (105339730627913794384744097333103439091751005834042565437622273746527120460296, 57526216366140097205440521137091723264218019353485851044212924740410855711206) - a point. This is probably the cause of the error. I looked at the createKeys() function, and it returns (private,public). However, when we call it:

 

They're the wrong way round.We were passing the public key (a point) as the private key. Once I fixed it:

 

Now we have a new error, and it was the error that I thought caused the other one. z is the wrong type - we need it to be an integer. At the moment, it's just a string. I replaced all instances of z = sha256(msg) with:

 

This should work:

 

Whilst it's not throwing errors anymore, it is also not the answer we want.

After a lot of checking, I realised that when we need to find , we use the invMod() function. However, if we look at invMod():

 

It returns x % self.p, whereas we want x % self.n. To fix this, I changed both invMod() and ecc() to take both n and p, where p is the value we want for the modulus. The new code looks like this:

 

This new code is more correct. Unfortunately, the verify function still fails.

 

This is very inconvenient to debug, as it is almost impossible to tell what the correct answers should be.

After hours of debugging, checking every single function - and rewriting half of them - I found the issue.

The problem was... a typo.

 

I forgot to change u2 to take r instead of z. Finally, the function works:

 

We can check that it rejects incorrect messages:

 
 

Conclusion

In this section, I built a working Elliptic Curve Digital Signature Algorithm in Python. This should allow me to easily convert to Javascript in the implementation phase.

I learnt a lot about the mathematics behind not just ECDSA, but also about finite fields and modular arithmetic.

From here, we need to design the application serves as an interface to Arbitra's functions.

Networking

We need to figure out how to ensure that the client can connect to the network. The way most other cryptocurrencies handle this is firstly maintaining a list of recent connections, and reconnecting to them. However, the client needs to connect to it's first node. Bitcoin, for example, solves this by having a few trusted nodes hardcoded into the client, which in turn maintain a list of a few trusted nodes. What I believe would be best for Arbitra is to create and maintain our own list, but of course allow the client the option of adding their own nodes. Bitcoin then also has some websites that maintain their own list of trustworthy nodes, which clients can connect to.

We also need a system of node discovery. The client is never going to expand it's list of nodes if it only connects to the default nodes. A way that we could implement it is by having mined blocks optionally include their address, so that whenever a new block is mined, a new IP is broadcast eventually to the entire network so that client's list of nodes can expand over time. However, having everyone connect to the same node if the manage to mine a block is probably not the best idea.

Bitcoin, for a time used to run an IRC server where nodes could broadcast their IP for people to connect to. However, this feature was removed as it was not scalable and provided a single point of failure to an aspect of the network. Whilst it clearly is not the best idea (as they got rid of it) it could be a stop-gap solution if clients are having difficulty connecting to enough nodes.

For Arbitra, I am probably going to have the websites with a list of trusted nodes, and then also have some hardcoded nodes in the client. The client will then remember nodes that it connects to. Then, most importantly, it will have a message type that will allow clients to ask other clients to share their list of nodes. This way, a client can expand their list

Application planning

Not only does the network need to be described in detail, but the application as well, as that is what will be used to interface with it. As previously mentioned, Electron will be used to create the application. Electron is a technology that embeds a webpage within a chromium instance, and also allows access to lower level OS functions.

This means that the UI is implemented in HTML/CSS, and the backend is done using Javascript.

There are three main parts of the application - creating transactions, viewing and interpreting the blockchain, and mining the blockchain. Therefore, the application should focus on these areas.

Something not yet mentioned is the ability to import and export wallets - users should be able to transfer their wallets from one computer or application to another.

Concept

The first thing I decided to do was to roughly sketch out how I wanted the application to look.

Concept 1

This concept was designed to be striking, with the purple backdrop drawing the eye. This was made thinking about how cryptocurrency wallet apps look like on mobile devices.

concept

Concept 2

This concept was inspired by this redesign of Windows File Explorer, by Frantisek Mastil.

windows file explorer

concept2

Concept 3

This is a purple/grey version of concept 2.

concept3

Pages

Key
pagepotential pagefeature

The first concept would be quite minimal, therefore there would only be three or so buttons that could be accessed from the main page. Therefore the structure of this app would be quite hierarchical.

overviewtransactionsminingsettingscreaterecieveimport walletshistoryview blockchainmineapp optionsnodesthemeview nodesconnect new node

However, the other two concepts are different. Since they both have a big menu on the left, most parts of the application can be accessed from every page. This makes the structure of the application much flatter.

any pagecreate transactionrecieve transactionimport walletstransaction historymine blockchainview blockchainnode optionsview nodesconnect new nodeapp optionstheme

Documented Design

How the network system works

The idea behind how I've structured the protocol is that there are different types of messages, each of which has a different specific reply. Each message is meant to stand alone by itself, and there is no 'conversation' between nodes. For example, if a node want's to check that it's blockchain is up to date, it will send out a message asking for the top block. If someone replies with their top block and it's different to what they have on disk, instead of replying to the same node asking for the blockchain, it will instead send out a chain request to all the nodes it is in contact with. This greatly simplifies complexity of the messaging system, allowing me to develop each function separately from each other.

A closer look at the Protocol

So that messages in the network can be understood by everyone in it, we need to make sure that there are strict definitions on message types. If a node receives an invalid message, it will reject it.

Since this project will be written in Javascript, each packet will use the JavaScript Object Notation (JSON). A rough example of a JSON object looks like this:

 

This means that it will be much easier to parse the messages as they are already in a format that Javascript can use.

However, it would be strange to have all of the info in one big group - for example, when checking a hashed block, you would have to remove some of the block's values and then hash it which, while possible, is inefficient and overly complex. The solution to this is separating each message into a header and a body. The header contains info about the block - who it was send by, what it's hash is, what kind of message it is etc. The body contains the actual message.

This system is far better because it means that the system can simply check the standardised header to find out what the message is, rather than parse each type of message separately. Also, since a lot of the messages are hashed, we can simply hash the content and put the hash into the header, which makes a lot more sense than having to remove elements from the message before you can confirm the hash.

A message with a header would look like this:

 
Pseudocode

We need to parse these messages, so I decided to plan the function that handles this in pseudocode.

 

This basically checks the message's type and calls the corresponding function. If there is an error, such as if the hash is invalid (indicating that the message has been tampered with) then it sets the reply to be an error message. Finally, regardless if an error was thrown, it sets the hash of the reply and returns the reply to be sent off.

Data types

So that it is easier to determine if a message is valid or not, we need to determine not only what the different messages are, but also the different formats that each part of the message will be (i.e. integer, hex string, enum). The way that I'll do this is with a table with the name, the type, and the maximum length.

The size does not apply to all types, mainly numbers. JS numbers are all 64bit double-precision floats, so it assumed they are that size.

The JSON object used in the first example above in this format would look like this:

nametypesize
idhex string64
senderhex string64
recipienthex string64
amountfloat 
timestamptimestamp 
signaturehex string64

Please note this is an example and not the final definition of a transaction message.

Header

Each message has a header. It has the following attributes:

nametypesize
typestring2
sizeinteger 
hashhex string64
versionstring 
timetimestamp 

size is the size of the body in bytes. hash is the SHA256 hash of the body, and acts as both an identifier and as a checksum. If size becomes a concern, it may become necessary to truncate the hash. The version is the version number of the client.

However, not all message types have unique contents, and therefore their hashes would be the same, making the usage of the hash as a unique identifier. I initially thought that this can be fixed by appending the timestamp onto the end of the body before hashing it.

 

However, this does not work for message types where they are relayed (block and transaction). This is because the timestamp is in the header, which changes each time that it's sent, as well as other complications that I ran into. I realised that a better solution was to move the time into the body. This way, the body is always unique, and the hash can always be found by just hashing the body.

The message type is a string, and is one of:

message typestring
Transactiontx
Blockbk
Latest block hash requesthr
Chain requestcr
Pingpg
Node requestnr

Message types

For each of the message types, I not only wanted to document it's attributes, but also what the client will do when it receives it, so I made flowcharts.

Transaction

A transaction, as mentioned earlier, has the sender, the recipient, the amount, the signature, and the time. However, it needs to be decided how to support multiple senders, so that in the case that someone need to send a large amount of Arbitrary Units, but had multiple wallets with that amount split between them, they would not need to clutter the blockchain consolidating funds before sending the transaction.

One way could be to have two lists, one with the recipient's names and one with the value they are given:

 

This is a good method as it allows you to clearly see the list of recipients.

Unfortunately, this is not very clear which amount is being taken from each wallet. It could also be vulnerable to the list's order being confused. It also does not support transaction fees (unless, for example, if the list of amounts is great than the list of recipients, the remaining amounts are given to the miner).

Another way of doing it could be to have a list of object literals:

 

A benefit of doing it this way is that it saves putting the total amount as a separate value. However, this then means that the ability to add a transaction fee is lost without having it as a different value.

Between the two, I would say that the second method is better, because it explicitly binds the recipients to the amounts they get. However, the other option is still available if, for whatever reason, I can't use the preferred method. I also aim to get rid of transaction fees, so not being able to implement that is fine.

Finally, a wallet is a public key, which is a point on the curve. We can convert them to hex and concatenate them into a big hex string, which should be 128 bytes long.

From this, we can make our table for transactions:

nametypesize
tohex string128
fromarray 
timetimestamp 

The from array would contain objects of the form:

nametypesize
wallethex string128
amountinteger 
signaturehex string128

The signing mechanism, as covered earlier, takes an input string and a private key, and produces a signature. However, the input string can't just be the amount, as then the signature can be reused. It also can't just be the amount plus the public key it's addressed to, otherwise an attacker could repeatedly send the same transaction. Therefore, we need to sign the amount concatenated with the public key concatenated with the current time. Since all of these are available in the transaction message itself, the signature can still be verified, but it also can't be duplicated.

Something to note about floating-point operations is that they're not very accurate.

 

This is not good for handling transactions like this. We can circumvent this, however, using integers, and then multiplying them by a fixed amount. I decided that the smallest unit an arbitrary unit could be split up into is a micro-Arbitrary unit, or . Then, we don't have to deal with floating point operations, as we can just use integers to define the number of au, which we can then convert back to au. For example, to send 50au, the amount would be written as 5000000.

The flowchart looks like this:

Created with Raphaël 2.2.0Receive MessageIs valid?Is in chain/txpool?Send to allSend "ok"Send "er"yesnoyesno
Block

Blocks contain transactions, which are objects. We also need to determine the method whereby the miner receives their reward, and the simplest way of doing that is to have a miner attribute in the body in which the person who mined the block can put any public key they desire, and it will be rewarded 50au.

nametypesize
transactionsarray 
minerhex string128
noncestring 
difficultyinteger 
parenthex string64
timetimestamp 
heightinteger 

The transactions array would contain the body of transaction messages. The nonce can be any string that makes the hash of the body meet the required difficulty. The difficulty is determined by how long it took to mine the previous block, but the mechanisms behind that works will have to be determined by testing once block mining is implemented. parent is the hash of the block that comes before it in the chain. height is the number of blocks it is away from the genesis block.

Created with Raphaël 2.2.0Receive MessageIs valid?Is in chain?Is parent in chain?Add to blockchainSend to allSend "ok"Send "cr"Send "er"yesnonoyesno
Ping

The ping message is a critical part of the network. Sending a ping signals that the node wants to be sent messages that other nodes receive. It also has a Boolean value advertise, which means that, if set to true, the node that received the ping will send the IP address of the node that sent it to nodes that send a node request. This should be a toggle that the client can switch if they receive too many messages.

nametypesize
advertiseboolean 
timetimestamp 

On receiving a ping, a client will reply with a ping. This shows that they acknowledge each other.

Created with Raphaël 2.2.0Receive MessageIs IP in connections?Send "pg"Add to connectionsyesno
Node Request

Sending a node request asks for the list of recent connections that each client maintains, provided that the connection has marked itself as willing in the ping message. A node request has an optional value defining the maximum number of nodes that it wants to receive (blank for all).

nametypesize
maxinteger 
timetimestamp 
Created with Raphaël 2.2.0Receive MessageGet connections where "advertise" = trueSend "nd"
Latest Block Hash Request

This is a small message that a client sends out to check that it's blockchain is up to date.

nametypesize
timetimestamp 
Created with Raphaël 2.2.0Receive MessageGet top blockSend "bh"
Chain Request

This function asks other nodes for the chain beneath the hash listed in the body. This is typically sent after a hash request.

nametypesize
hashhex string64
timetimestamp 
Created with Raphaël 2.2.0Receive MessageIs hash in chain?Get chain below hashSend "cn"Send "er"yesno

Reply types

Not only are there messages that are sent out by the client, but there are also the different types of message that are sent back in reply to these messages. Some we have already touched on, such as the block and the ping. However, some messages don't need a specific reply, so they have a generic "received" message. Each reply corresponds to a message:

namereply tostring
Pingpgpg
Chaincrcn
Block Hashhrbh
Nodenrnd
Receivedtx, bkok
Ping reply

The ping reply is just another ping.

Chain

In reply to a chain request. It is just an array of blocks block and it's associated hash.

nametypesize
chainarray 
timetimestamp 

When a client receives this message, it verifies each one and if it passes, it adds it to the blockchain.

Block hash

In reply to a block hash request. It is simply the hash from the top of the blockchain.

nametypesize
hashhex string64
timetimestamp 

When a client receives a block hash, it checks it to see if it is the same as their top block. If it is not, it sends a chain request to all connections.

Node

In reply to a node request. This is simply an array of nodes from the list of recent connections that the client maintains that have marked themselves as willing to be broadcast across the network.

nametypesize
nodesarray 
timetimestamp 

The array is just an array of strings.

When a client receives this message, it sends a ping to each of the IP listed in the message.

Received

The received message type is just a confirmation that the message had been received and accepted. Therefore, the only thing in the body is the timestamp.

nametypesize
timetimestamp 

Error Message

As well as correct replies, we need to messages that a client will reply with if the messages received is incorrect in some way. The message will have type er, for error. It also contains the hash of the failed message, if available.

nametypesize
errorstring20
hashhex string64

The error can be one of several strings, that correspond to different errors.

error stringdescription
parseFailed to parse JSON - message is not valid JSON
hashHash does not match
signatureSignature is invalid
typeThe type of the message is invalid
amountTransaction invalid due to not enough funds
transactionTransaction in a block is invalid
notfoundRequested block/data not found

Network diagrams

In order to gain an understanding about how the network would function, I decided to model a network with a limited number of nodes, manually.

S1S2S3S5S4S6

I set up the network in this way so that network traffic has to disperse through the network through multiple nodes. This simulates a real network, where not every node will be connected to every other node.

The rule is that when a node receives a message, it passes it on to every node that it is connected to.

The arrows indicate that the message is passed from the node on the left to the node on the right, and the lines indicate a step. The nodes listed after the line are the nodes that have received the message

Simulation 1: From S1
 

I'm going to stop the simulation at the second state, because I can see a loop is starting to form. Since S2 is connected to S1, it will send the message back, which in turn will send the message back, forever.

This can be fixed with a single rule: No passing messages back to the node that sent it.

However, there is another issue - in a loop of three nodes like S1, S2, and S3, the can still be a loop if a message gets passed from S1 to S3 via S2. Since S3 doesn't know that S1 sent the message, it can send it to S1 which will send it around the loop again. Therefore, we need a second rule: Never send the same message to the same node twice. This, in the actual network, would probably mean keeping track of the list of messages that have passed through the node via their hash, and not repeating an incoming message if it's hash is on this "blacklist".

Simulation 2: From S1
 

User Interface Design

Since the application will use Electron, the frontend is made with HTML/CSS. Because I have experience with web design I decided to prototype the UI using a static HTML page.

The code used to create the UI is located at the bottom of this section.

Concept 1

I decided to go with concept 1.

Initial Prototype

This first pass, while it captures the aesthetics that I'm going for, does not have the functionality I want. There are no buttons, and I want a visually striking graph or other graphic to fill the space beneath the current balance and the important buttons.

Updated Prototype

In this updated prototype, I remove the placeholder text, as there is no need for text anyway, and moved the existing information boxes to the bottom. I then added the most important buttons - making a transaction, mining the blockchain, and settings. Beneath those is a temporary graph (which can be found here). The proper graphic, whatever it turns out being, will be the same grey colour with purple highlights.

This update is an improvement, but it is still missing some features. The most important being a way to access the transaction history and other minor features. However, I don't want to clutter the top of the page. In a normal website, this would probably be hidden in a sidebar menu or something similar, but since this is pretending to be a desktop app that would be out of place.

The solution I came up with was to create a preview list of previous transactions. Through this approach, there is more information available at a glance to the user, it adds more functionality to the homepage, and it creates a non-intrusive way to add the link to the list of previous transactions.

Arbitra UI update 3

I think it works quite well. You would be able to switch between all recent transactions, all recent outbound transactions, and all recent inbound transactions, and also be able to navigate to a more in-depth page.

I also made some slight adjustments to the spacing to make it more consistent, as well as adding a 0.5 second easing animation effect to all background transitions using transition: background-color 0.5s ease.

Something that occurred to me when using other cryptocurrencies is that it is often unclear when the application is downloading the blockchain. I therefore had the idea to turn the purple banner at the top of the application into a huge loading slider, like so (mockup):

Downloading concept art

I had to manually position the 45% as it was a child of the progress bar itself, so this is obviously a temporary solution. However, if I have time it would be a cool way to incorporate the banner into the functionality of the application.

Redesign

I really don't like the way this it turning out. It feels more like a website than an app, and with this in mind I decided to try to rearrange it into concept 2, from the analysis. With this second attempt I took into consideration:

redesign

I think that this is a vast improvement. It now looks much more like a traditional desktop application. The colours are much more professional and the way you navigate through the app is much more streamlined and easy to understand.

I also made the drag region a separate div. I highlighted it here in red:

highlighted drag region

After some minor polish, this is the final concept:

final concept

Source code

HTML - index.html
 

The HTML structure is split into three parts - the left menu, the drag bar, and the main section. They are given the class names .left, .dragbar, and #body, respectively. #body and .dragbar are grouped into a div called .right, to ensure that they are kept together.

The menu in .left is made up of a series of divs called .subsec and .items. There is probably a better way of doing this, but I was struggling to align list elements so I decided to use divs. .subsec describes the menu subsections, and .items describes the different links - these will have to be buttons eventually, but for now they are fine.

The drag bar is made of two elements - .dragbar, which is what you can use to drag the window around, and .closebox, which contains minimise, maximise, and close buttons - they are now Font Awesome icons. This is because if the entire top section is a draggable region, you are not able to click the close buttons. To solve this, I made sure that the two elements do not overlap.

CSS - style.css
 

For the CSS, I made use of some cascading, for example making all the icons in the .closebox div have the same colour, size and padding, without having to give them their own class.

I also used the calc() function a lot as it allowed me to, for example, make the .dragbar full width minus the size of the close buttons.

To get the "au" suffix on the number on the top, I used a ::after tag with content: "au"; and a different size. This meant that every element with the .money class is automatically followed by "au", which is very useful as if I have a text element that I know will contain an amount of currency, I can give it this class rather than having to manually append "au" in the backend.

For the colours, I generally used #333 (grey) and #fdfdfd (off-white). The font is Segoe UI.

Generic Page Template

Now that we have the overall style decided upon, we now need to flesh out all the UI elements that we need. This includes:

Since the page is pretty much already blank, all I did was create a new file and copy the HTML code over.

To start, I listed out most of the HTML elements that we might need.

 

testing page

As you can see, since I removed all padding and margins from every element at the start of the CSS page, the p elements lack any separation between paragraphs, and the lists are not indented as they should be.

Furthermore, the buttons are very ugly by default in Chrome, and do not fit with the style at all.

However, they all inherit the correct colours from the #body div, which is what we want.

My first attempt to fix this was to add back the default padding and margins.

broken margins

This completely broke it, as many HTML elements have default margins that I was relying on not existing. However, if the margin is set to 0 but we don't change the padding, it fixes the unordered and ordered lists.

fixed margins

The next step is to fix the p tags.

 

improved p

At this point, I think that basic elements like headings and paragraphs are fine, so I will remove them from the testing page. The next thing to do is to try to style the buttons, as they are an important part of the visual theme of the application.

 

buttons

I decided upon a capsule-shaped button with very rounded corners and a thin, 1px thick border. The second button is what it looks like when hovered over - it transitions to a dark background to signify to the user that it is a button. The transition is achieved by applying transition: background-color 0.5s ease and transition: color 0.2s ease to * (all elements).

I also applied margin-bottom: 8px to h1 as it was touching the text field.

 

text fields

I found that much of the styles applied to the buttons could be also applied to the text field. As with the buttons, the second field ("Number field") has been focus. As you can see, the focused field as slightly less rounded corners, so that it is clear which field has been selected.

I am not 100% happy with how it has turned out, but it is fine for a first pass.

It turns out that to create custom radio buttons and checkboxes requires replacing them completely, and since I don't think that I'll use them much they are fine having the default look.

At this point I added a hyperlink to the testing page, and changed the max-width of the text fields to 278px so that they are the same size as the dropdowns.

 

highlighted area and hyperlink

For the highlighted area I decided to go with a #333 background and rounded edges, and for the hyperlink I decided to just lighted it slightly and keep the underline to differentiate it from the rest of the text. I also used p > a to select it, as it then only selects hyperlinks that are in text as otherwise it could accidently apply the background and padding to a div or button that is wrapped in a hyperlink, which would cause issues.

 

a hover

Using the :hover pseudo-class, the hyperlink has a dark background when hovered over with the mouse. I used negative margins to ensure that the increased padding did not change the size of the element, as otherwise it would shift around on the page when hovered over.

Lists

There are multiple instances in the application where we will need to display a list of information. I decided to implement this using HTML tables. Here is a basic table with some arbitrary data:

 

As you can see, the hashes are very large. We need to make sure that it doesn't push the rest of the contents off of the screen. In this case, tables may not be the best option. Perhaps it would be better to simply do it using divs, as then we can make sure it is responsive.

list 1

There is also the issue that transactions can have multiple recipients. To account for this, I created a list object that would be repeated. I also used wallets instead of transactions, but we can reuse the same styles on the transaction page.

 

This produces:

wallets

This is much better looking, and much more flexible than the table approach. It is much simpler than the table, consisting of an outer div, then the .list div which contains all the .list-items. .list-item is very simple, as it just contains three p tags on top of each other.

 

Creating the Electron app

In order to start we need to create the Electron app. Since this is the first time I have used Electron, I followed the Electron Quick Start guide:

https://electron.atom.io/docs/tutorial/quick-start/

First, we need to create the package.json file. This contains information about the application and where the main Javascript file is, in order to run it.

 

We then need to make the main.js files in order to create the window. This is mainly boilerplate code which won't change for our purposes, taken from the Electron Quick Start guide.

 

Then we need to include the HTML and CSS files. For this I used the HTML and CSS code from the concept in the UI Design section. I named these files index.html and style.css, respectively, and placed them in the same directory as the other two files.

Finally, we need to install Electron. Since I already had Node.js installed, I used the Node Package Manager to install it. After navigating to the directory where the rest of the project was using cd <filename>, I used the command npm install --save electron. Using the --save tag ensures that it is saved in the node_modules directory it creates.

installing Electron

It successfully installed. That image is just the top section of the install process as it outputs a large amount of irrelevant data. All that is left is to start up Electron. According to the Electron Quick Start Guide, the command to do this (from the directory) is .\node_modules\.bin\electron ..

running the command

A few seconds later, the window appeared!

arbitra running

Removing the Frame

The default "chrome look" of the window is quite ugly. Luckily, it is quite simple to remove. We simply change

 

to

 

This changes the window to look like this, which is a vast improvement.

removed frame

However, we now can't drag the window around, or close it. We need to be able to use the top area to move the window, and implement our own minimise and close buttons. For the time being, I will just implement the close button to avoid needing to find icons etc.

Luckily, dragging the window is easy - we add a property to the CSS called -webkit-app-region.

We want to make the purple area draggable, so we add -webkit-app-region: drag to .dragbar. This works, but I can't really demonstrate this with an image.

We now need it to close when it is clicked. My first attempt was to add the function call onclick="closeWindow()" to the HTML, and in main.js add the closeWindow() function:

 

However, this does not work. It turns out that, because Electron is based on the Google Chrome browser, there are two types of process: the main process and the renderer processes. The main process is like Chrome itself - it is a kind of background process. Then, each tab has it's own renderer process. This is why when you close a Chrome tab it doesn't close down Chrome itself. In our app, main.js is the main process, and in order to interact with the window to close it and do other functions, we need to create a renderer process for the main page. This file is the equivalent of the client-side Javascript that it normally used in front-end web development.

Therefore, I created renderer.js, and imported into the HTML at the bottom of the body.

 

In renderer.js, we need to import a module in Electron called remote. remote is how to interface with the window. Then, we make the document call a function when the "ready state" changes. This function checks if the ready state is ready - effectively, this means that everything inside the if statement will only be called when the document is ready. This is necessary as if we try to select an element before it is ready, it would not work, meaning that it could leave the close button without an event listener

 

With that done, we can add the event listener, which uses remote to get the current window, and then closes it.

 

This works - we can now close the app using the x.

We can use this to create minimise and maximise functions:

 

The minimise function is simple, as it simply calls window.minimize() to minimise it, which is a function already in Electron. However, it is less simple for maximising the window, as the same button must also return the window to it's old size. For this we call window.isMaximized() to see if the window is maximised. If it is, we call window.unmaximize(), otherwise we call window.maximize().

The solution to this problem used this site: http://mylifeforthecode.com/making-the-electron-shell-as-pretty-as-the-visual-studio-shell/

Changing pages

We obviously need to be able to change pages in the app, and so for this we will be creating a testing page. I copied the HTML code to a new page called testing.html. I then added to .left the following code:

 

This should, when clicked, change the page from index.html to testing.html.

And it does. However, there is a problem, in that the app freezes and goes completely white for a second while the new page loads, which is unacceptable. This is because Electron is intended to host Single Page Applications, which load all the content at once then swap pieces out instead of completely reloading the page. This means that we need to create a system for doing this. Luckily, the app is set up in such a way that we have a div that we can change, and then keep everything else the same.

As it would be a real pain to keep everything in the index.html file and hide/unhide sections as we need them, I think that the best way of doing this would be to keep each page in a separate HTML file and then import them into index.html as needed using Javascript.

We can use the innerHTML() function in Javascript to do this. It takes a string, and when you apply it to an element it sets the children of the content to what the string is.

Before we do this, we need to set up the page we intent to change. First is the Overview - the main page. I created a new file called overview.html, and put it in a new directory called pages.

 

I also changed testing.html and moved it to pages.

 
Pseudocode

I decided to plan this function using pseudocode.

 

Now, we need to create the function that changes pages. In renderer.js:

 

First we import the file system library, from Node.js, and set it to fs.

The changePage() function takes in the name of the page it changes to. It then creates the path name of where the pages are kept. It then uses fs.readFile to attempt to read the file.

If there's an error, it prints the error both in an alert and in the console. Otherwise, it prints that the page is changing, and then sets the innerHTML of #body to the data that that fs.readFile read.

However, there is a problem. When I tried to run this code, it threw an error.

fs error

This is because it tried to read the file C:\pages\overview.html rather than arbitra-client\pages\overview.html . We can fix this by changing

 

to

 

This makes it a relative path rather than an absolute path, and so should now link to the correct file.

fs working

Trying again, the application loads properly. This means the function works. We now need to turn the .items in the menu to buttons that change the pages. We can implement this using document.getElementById(). () => {...} is "arrow notation" for a function - it is a shorter way of writing function() {...}.

 

I gave every menu item an ID, so that they can be selected.

 

When we click the Testing button, it changes page. This means that the navigation works, and we can massively simplify the process of creating and changes pages.

changing page working

I also created a folder called js, where we can keep per-page Javascript files. Then in the changePage function, it imports the page's JS file, then calls a function which initialises the page. We use exports to expose functions in the file, in this case init(). In each per-page JS file (for example testing.js):

 

Then in changePage():

 
 

It works. This means that we can properly separate the Javascript code into files.

Highlighting Menu Links

I noticed when implementing the menu that the highlighting on the menu items was inconsistent, as it only applies to the .items and not the .subsecs. However, not all the .subsecs are divs, which means that we can't just apply the highlighting to them as well. We can simply fix these problems by giving all the .items and .subsecs that are meant to be divs a new class, called .link. We then give the highlighting properties to the .link class.

Replacing .left > .items:hover:

 

I also added cursor: default to .subsec. This means that the cursor when hovering over the menu items is consistent depending on whether it is a link or not.

The crypto Module

crypto is a default module within Node.js that provides cryptographic functions, similar to Python's hashlib. They both provide wrappers around OpenSSL, which means that translating Python code from the experimentation phase should be fairly simple.

The first test is to hash a string using SHA-256. In order to do this in Javascript we need to first import crypto into the project. In main.js:

 

This imports the crypto object and sets it as a constant.

The next step is to hash a string. Using the documentation, I wrote this code to hash the string "something":

 

According to the experimentation code in Python, this should yield the hash 3fc9b689459d738f8c88a3a48aa9e33542016b7a4052e001aaa536fca74813cb

 

It produced the same hash, which means that it works. Now, it would be good to put that into a function so that we can call it whenever we need to hash something. I also moved the function to renderer.js, as that is where it would be called.

 

As you can see, I've compacting the hashing to one line - it is still quite readable, and of course produces the same output.

Networking

A fundamental part of Arbitra is the Arbitra Network - the application by itself means nothing if it cannot connect to others. This means that we need to implement a reliable peer-to-peer networking system.

The way Arbitra will implement this is using the net Node.js module. From it's documentation:

The net module provides you with an asynchronous network wrapper. It contains methods for creating both servers and clients (called streams).

Because the internet is centered on a client-server model, where clients request and receive data from a central server, most of the Node.js documentation and functions were focused on this.

However, even though we want a peer-to-peer network, we can still use this model. The client can run a server, and when another client wants to send it data it can connect to the server as a client. The first client can reply in this connection, but it will then be cut off. If the first client wants to send something else to the second client it connects using their server.

We can simulate this over localhost, the computer's local network. We don't even need to run the client and server in a separate file in order to simulate the model. The two upcoming snippets are in renderer.js.

Server Example

 

First, it imports the net module.

It then creates a server. The server has a callback function which adds event listeners to the socket.

A socket is the term for the endpoint of a communication stream across a network. If it is bi-directional (this one is), it can send data down the stream both ways.

The socket is how we interface with the data that the server receives. We can add an event listener using the .on() function. When the socket receives data, it will call the callback function applied to the data event listener, which in this case prints to the console whatever data that it receives, then hashes the data and sends it back down the stream to the client. In actual use, it would pass the data on to be processed. It sends the data back using socket.write(hashed).

There is a second event listener that ends the socket when it detects an end, to make sure the socket on the sever side ends when the .

Finally, we call server.listen(), which starts the server listening on port 80. We give it the parameter 80, which is the HTTP port (to ensure traffic is not blocked). This means that the server will reply to all traffic on the network that is in port 80.

Client Example

 

The client creates a socket which we connect to the same port as our server, and the IP 127.0.0.1, which is the loopback address, meaning it connects to itself without leaving the machine. When it connects, it sends to the server "hash this string please" using client.write(). Using an event listener once again, it prints to the console whatever data that it receives, then destroys the socket. It then uses another event listener to detect that event and print "Connection closed" to the console.

Running the program creates this output:

client/server model

This simply model clearly demonstrates how we can use the net module in order to send data from client to client.

However, we need a system that determines what messages need to be sent. I made some flowcharts to map the logic that the program should follow:

Created with Raphaël 2.2.0StartAre there any nodesin the connection list?Attempt connectionDid itconnect?EndRemove nodeGet nodes fromhardcoded serverDid anyconnect?Throw erroryesnoyesnoyesno

When it connects, it will then go through this flowchart:

Created with Raphaël 2.2.0StartAsk for latestblock hashIs local copyup to date?EndAsk for for missing blockyesno

Then there are also the other message functions that the client can receive, which are:

Data Files

The is a lot of data that we will need to store. The way that we do this is by storing it in the applications' %APPDATA% folder, which we can access using the remote module.

 

We can then store data using the fs (File System) module, which will be covered during the Technical Solution phase.

Files will be in JSON (JavaScript Object Notation), because as the name suggests it is directly compatible with

File Planning

There are several different files that we need to have. The obvious ones are listed here:

NameDescriptionType
blockchain.jsonThe blockchain{}
connections.jsonList of nodes that are currently connected[]
recent-connections.jsonNodes that have been connected to[]
sent.jsonSent messages so they aren't resent[]
network-setting.jsonSettings options{}
error-log.jsonStores error messages[]
txpool.jsonPending transactions that are used to generate blocks[]
wallets.jsonStores wallets[]

The type indicates if the file will be in Array form or in Object literal form.

Technical Solution

Converting Python Code

ECDSA

First, we need to work out some of the differences between Python and Javascript. First of all, and most importantly, Javascript only supports numbers up to , which is an issue considering , a significantly larger number. Whereas Python handles this automatically, we need to use a library to handle the large numbers that we have to use. I chose https://www.npmjs.com/package/big-integer, as it has a mod() function, which others that I looked at lacked.

Installing via npm:

 

We then require() it at the top of the document.

 

Secondly, JS has no tuples. Therefore, we need to represent points as an object literal.

 

We can then get the value by calling point.x to get 20. JS also has support for Infinity, which we can use as the point at infinity.

Thirdly, while JS does have classes, it would probably be better to just have the curve characteristics as a const and not have the functions in a class. The curve therefore looks like:

 

When we want , for example, we would call curve.p.

I put curve and sha256() into a new file called ecdsa.js, in the /js/ folder, which will contain all our ECDSA code.

Fourthly, we want to make sure that this code runs asynchronously. This means that that the rest of the client can do whatever it wants, while this signature code runs for as long as is wants without disrupting the rest of the program. To do this, we use callback functions. These work by instead of returning a value at the end of a function, we instead pass a different function to the the first function which it then passes values to.

Finally, we need random numbers

In Python, we used random.randrange to generate the private key and . However, there is no equivalent function in crypto. We could use crypto.randomBytes() to determine the secret key, as it claims to be cryptographically strong. However, since it needs to be less than , a naïve approach might be to % curve.n.

 

However, values more than would wrap around to be small numbers again, which produces an uneven distribution of random numbers, which is not random. While there are modules to do this, we're going to have to simply generate random numbers until they are less than .

 

Note this function can be passed minimum and maximum values, but default to and , as that is all we need the function for.

sha256()

We need to convert the sha256() function we made earlier to return bigInts. This is relatively easy.

 
onCurve()

This one is pretty simple to translate. However, we have to use big-integer, so that means using .minus() instead of simply - etc.

 
invMod()

big-integer has an modInv() function which is identical, but is designed to work specifically with bigInts, so I decided to use that instead.

addPoints()

Now we have the other functions, we can implement addPoints(). Due to big-integer, it looks incredibly messy, although I tried to separate out the expressions a bit

 
multiPoints()

With multiPoints(), we need to convert a number to binary then iterate through it. First of all, we need to get the binary number. Luckily, when we use .toString(), we can specify the base, in this case .toString(2). Otherwise, the function is very similar to the Python version. yranib is the word "binary" reversed.

 
createKeys()

Now for the key creation function. Here is where we use randomNum().

 

However, createKeys() itself should be an asynchronous function, since createKeys() takes time to run and things rely on it. Therefore, I restructured it.

 

This is now much simpler and should work much better down the road.

signMsg()

signMsg() also uses callbacks. It also catches any errors in the multiPoints() function, which would appear if w was an invalid number, and return a error to the callback. I also restructured the function around a while loop which is much easier to understand.

 
verifyMsg()

verifyMsg() is also similar to it's Python equivalent

 

Finally, we can export the functions that the rest of the program will interact with: createKeys(), signMsg(), and verifyMsg(), as well as curve in case we need to reference the curve attributes somewhere else in the program.

 
Testing

In testing.js, I added the following to init():

 

This returns:

 

This appeared to work. We now need to test the message signing/verification to confirm that. I changed init() to this:

 

This creates keys, signs the message "bean", then verifies the message, throwing any errors it picks up along the way. This returns:

 

This means that it works. We can now use ecdsa.js to sign and verify messages.

Simplifying Functions

While the mathematical aspect of the function works, it is inconvenient to call from other parts of the program, since all the keys are in bigInt format within ecdsa.js, and are as hexadecimal strings everywhere else. Therefore, we need to change these functions so that they take in and return values that are compatible with the rest of the program. I started with createKeys().

 

Now it returns hexadecimal strings, so the rest of the program can take that output and use it without conversion.

Next we need to convert signMsg(). This is relatively simple, as it only took one bigInt as an argument. However, we also need to turn the signature into one hexadecimal string.

 

Finally, the last outward-facing function is verifyMsg(). This is more complex, as we need to parse the signature,

Hashing

We have already converted the sha256() function from Python in the ECDSA section of this document. However, since it is an important function, I put it in it's own file, hashing.js, which is located in the /js/ directory. I then created two functions, one which returns a bigInt, and another which returns a hexadecimal string, and also set up the exports function.

 

Now we can import our hashing function from all over the application.

File

As mentioned previously, to avoid messages going in circles we need to store the hashes of received messages so that we don't resend it to people. For this, we will store in a JSON file the hash of the sent messages, along with the IP addresses that it has sent them to. The file will be in the following format:

 

By using the hash as a key, we can simply use file[hash] to retrieve the array of addresses that that message was sent to, which is far easier and more efficient than having to search for it.

The method by which you read and write to files in Node.js is using the fs (file system) module, a default library. However, it is powerful but quite clumsy, so I created some functions that wrap around fs' functions that are better adapted to this project's needs. These are stored in file.js.

store()

First we need to import fs and electron.remote into file.js, since we need remote to get the file path of the %APPDATA% folder where the JSON files will be located.

 

Then we make a new function called store(). This will open the JSON file, append a new hash, and then save it. We get the correct file path using remote.app.getPath('appData'). I had originally designed this function to just work for sent.json, but this is changed later.

 

Then we need to get the contents and JSON.parse() it, then add our new data.

 

We then rewrite content to the file.

 

However, this will throw an error if sent.json doesn't exist. We can fix this by catching the corresponding error (ENOENT) and creating a new file if that's the case. We do however have to put content into an array so it is in the correct format.

 

However, this doesn't quite work, when we call the function with the following input:

 

It creates sent.json, as expected...

 

However, the actual contents of sent .json looks like this:

 

This is an easy fix, as I simply forgot to JSON.stringify(content) when creating the file. However, it does raise the issue about how to deal with invalid data when opening sent.json. I decided restructure the function so that it would deal with unexpected errors better.

 

Now, the function opens sent.json, checks if there's an error, and if there's not it tries to parse the content. If it doesn't work, it uses an empty object. Finally, it puts the data into the object, turns it back into a string and writes this new string back into the file. If there is an error when reading the file, it checks to see if the error corresponds to the file not existing - if that's the case, it sets the content to '{}', an empty object. Otherwise, it will throw an error.

It was at this point I realised that we will want to use this in other situations as well, such as to store wallets or the blockchain. I therefore made it take a third argument, file.

 

Finally, we need to consider what happens when there the key already exists. We need to add the new data to the old data. To do this, we need to merge to two arrays. To do this, I found the best method was to first concatenate the arrays, then them into a set turn it into a Set then back into Javascript. We turn them into a Set in order to remove duplicates, as a Set will only accept unique values.

https://gist.github.com/telekosmos/3b62a31a5c43f40849bb#gistcomment-1826809

 
Removing arrays

If we want to make it truly generic, we need to get rid of the code relating to arrays, or at least put it in a separate function. I decided to solve this by adding a check to see what type data is. If it's an array, we do the concatenation thing with Set, otherwise we just replace it. I originally tried to do this using typeof, however upon testing, it simply returns "object" for arrays, which is a problem given that we want to differentiate between array objects and other objects.

 

However, I found out that we can simply use Array.isArray(), which returns a simple Boolean. I put this in the if statement with jsondata.hasOwnProperty(key), so that it only does the concatenation thing if both the key exists and the data is an array.

 

Finally, I added an optional callback that is called when the file is written to. I did this by giving callback the default value of ()=>{}, which is ES6 arrow notation for a function, meaning that if no argument is passed for callback an empty function is called instead.

 
Testing

I tested this by calling store('test',['data1','data2'],'test'). This returned:

 

In the file C:\Users\Mozzi\AppData\Roaming\arbitra-client\test.json.

Next I called store('test2','hello','test').

 

Strangely, the key 'test' has been replaced by 'key'. I guessed that it was an error relating to using {key:data} if there was an exception, so I replaced all instances of that with:

 

and ran the same tests again. This ended up with:

 

and then:

 

We can now test the array concatenation. Calling store('test',['data2','data3'],'test') gave:

 

It worked!

get()

Now we need to retrieve data. This function is very similar, except instead of writing to the file at the end, it just calls the callback with the data it retrieved.

It also returns null if either the file or the key doesn't exist, which can be dealt with in the callback.

 

However, I noticed later on that returning null if no data is found was awkward to work around. I therefore added fail, which is an optional parameter which is returned if no data is found.

 

I also now exported the functions, so they can be used in other files.

 

I realised that it would be helpful to add some more functions to deal with other problems. These ended up being:

getAll()

This is simply stripping the complicated parts out of get().

 

storeAll()

storeAll() is even simpler, as it just wraps fs.writeFile().

 

append()

Append is very similar to get(), but rather than dealing with objects that sometimes have arrays as data and using Array.isArray(), it simply appends the data to the file using jsondata.push() as the file is assumed to be an array rather than an object literal.

 

Message Sending/Receiving

In the design phase we figured out how to use the net module to set up a TCP server. Now, we need to create a system where we can send and receive messages in the background of the application, all while the rest of the app does it's own stuff. First of all, I put the code from the design stage in it's own file, network.js. We can them import it into renderer.js to run in the background of the application.

 

Then in renderer.js, after it calls the changePage() function, I put this:

 

This produced:

 

This is promising. We can run the sendMsg() function from anywhere and the server will reply. Now we need to structure the server so that it can reply as we want it to.

 

This passes received data to the parseMsg() function, which should return a reply which the server then sends back. This will be created later. For testing purposes, it simply returns a hash of the input message.

 

Since we'd never be sending one message at a time, I created a function that opens connections.json and sends a message to each of the IPs within, sendToAll(). It uses a function file.getAll() which I haven't created yet, but in summary it opens the file with the name passed to it, and returns the data in that file. This function calls file.getAll() to get connections.json, then iterates through the objects in the file using forEach(), and sends a message to each of them.

 

But what causes clients to start sending messages in the first place? We need to make the system where it figures out the data it needs and starts asking for it without prompting. When the app starts up, it needs to do the following:

I decided the best place for this is within the network.init() function, after we set the server running. First of all, we need to wipe connections.json, as this is meant to contain current connections and it is unknown if those connections are still there. Therefore, we remove it's contents when the application starts. This uses file.storeAll(), which stores an empty array.

 

Next, we need to connect to the nodes listed in recent-connections.json, to reaffirm that they are still around. I decided to put this in a function called connect().

connect()

This is a fairly simple function, as it simply gets all the nodes in recent-connections.json and sends them a ping, if they're not currently connected to. However, I also wanted it to connect to the backup server if it couldn't find any connections. Therefore, after it sends the pings I wanted to see how many connections there are, and if there's none it should attempt to connect to the backup server. First, I ran into the issue of knowing how many connections I had. Initially, I tried a complex callback system but this ultimately failed.

I realised later that I could just use the connection counter in the top left corner. This is increased every time the client receives a ping, so if we read that we can tell how many connections the client has.

The second issue is that pings don't come back instantly. Therefore, I used the setTimeout() function to check the number of connections after 10000 milliseconds have passed.

Finally, the backup server is 5.81.186.90. I also show the "WARNING: No connections" message until the ping has returned.

 

setInterval()

As per the list, the next thing we need to do is create a loop that sends out hash request messages and pings, where neccessary. Rather than use a while loop, I found that a better way of doing it was by using the setInterval() function, as then we can set a gap between the loops. I chose 60 seconds, or 60000 milliseconds.

What this function does it get the number of connections from the DOM, as mentioned previously, and also get the target number of connections from the settings file. Then, if the number of connections is less than the number of target connections, it calls the connect function again. It then waits 15 seconds for that function to finish, then gets the number of connections again to see if it's over the threshold yet. If not, it sends out node requests to the connections in an attempt to get more nodes.

 

I also wanted it to send out hash requests, so I added that. On top of that, if connections is greater than the target, it needs to replace recent-connections.json with connections.json.

It only sends out hash requests if there are more than one connections. If there are no connections, it removes the .hidden classs from the "no connections" warning.

 

Message Parsing/Processing

Once a message has been received, we need to parse it and send it's information on into the system. We also need to ensure that a message is valid, as one of the critical parts of a cryptocurrency is ensuring that all messages are correct and valid.

First, I created an example message, that is a transaction.

 

This is a JSON object that represents a transaction. Whilst it is not at all correct, we can use it for testing the system as we build it. The function that the server calls is called parseMsg(), and it takes the received data and a callback function. First of all, we need to be able to store sent transactions.

parseMsg()

The best way of doing this that I can think of is having a function for each message type. First, we need to create the if statement that handles this. First, it creates an object literal for the reply. It then attempts to parse the message into JSON. If it doesn't parse, it catches the error and calls the er() function, which takes the error message, which should set the reply to an error message. Otherwise, it checks the header type and calls the corresponding function. After the reply has been constructed, the header is created and then the reply is turned back to a string. Finally, it is passed to the callback function where it will be sent back to the sender.

 

First however, we can simplify this system slightly. Every message has a hash, so we can verify that once before we start parsing the messages. Therefore, I added the check before the if statements. If it fails, it will call er() with the error message of 'hash'.

 

I also realised that we could use the try...catch statement to set the error reply, rather than using er(). If it runs into an error, it simply throws the error, which will then be caught by the catch statement. By throwing what we want the return message to say, we can simply rewrite the try...catch statement like so:

 

We also need to find the size of the body. I found an answer on StackOverflow that answered this question, using Node.js' Buffer - Buffer.byteLength(string,'utf8'). We put this at the end of the function.

 

Parsing Different Message Types

To make things a bit clearer, I decided to split these functions into a new file, parse.js, so that parseMsg() is more readable. However, this means that each of the functions like tx() will be converted to parse.tx().

I imported the following files, as all are needed.

 

The first of these functions is pg(). Since we need the IP of the node that sent it, we pass the IP as well as the message.

pg()

What we need to achieve with this function is:

However, since we need to do the first part again when we receive a ping as a reply, I decided to split it up into two functions, one which adds it to connections.json, and the other which replies with the ping. pgreply() adds the IP to connections.json, and pg() calls pgreply() and then returns the reply message.

First, I created pgreply(). First, it creates the object which it will store later on. Then, it gets all the connections using the getAll() function I made. It then iterates through the connections, and sets repeat to true if the IP that sent the message is already in connections. If repeat is false, meaning that it is not already connected to that node, then it appends the store object, which contains the IP and whether or not the node wants to be advertised.

 

However, since we don't want to connect to ourselves, I decided to install the ip module and check if our IP is the same as the one we would be adding.

 

We then check if it's the same

 

Back to pg(). I created a new file in %APPDATA% called network-settings.json, and then added the following:

 

This will be automatically added later on.

We then get() this file, then set the reply message accordingly. It checks to see if data is either 'true' or 'false', as I was having issues with that at an early stage.

 
Testing

I decided to test the ping function by creating function in testing.html to ping an IP. testing.html looks like this:

 

testing.js is changed to this:

 

It creates a message, msg, then when the send button is clicked it sends msg to the value of the input box using network.sendMsg().

Hopefully, if we send this ping to localhost we should see it print the same message twice, as the reply should be the same as what we sent.

 

This looks mostly good. However, I noticed that in the reply, it doesn't reply with a value for advertise, which is a problem. Looking again at the code for pg():

 

The issue lies in file.get() - since the data is returned in a callback, and since callbacks are asyncronous, reply.body.advertise is being set to data after the reply is returned. My first solution was to move the return reply into the callback.

 

However, since the callback is itself a function, this is simply returning reply to where it is called in file.get() rather than returning it back to parseMsg(). I struggled with this issue for quite a while, as I didn't know how to get the data from file.get() to return syncronously.

Restructuring parseMsg()

The solution, it turned out, does not use returns. Going back to parseMsg(), the callback is effectively a function that sends a reply. Therefore, rather than getting each file to return a reply, it is better to pass callback() to each of the type functions. Then, it creates a reply and passes it to the callback, rather than returning it back down to parseMsg()

In this way, the whole process is completely asyncronous as at no point is the program waiting for a function to finish.

The redesigned parseMsg() now catches any errors and appends the received data to a file called error-logs before sending a reply. It also sends ip to pg(), which will be passed to the function when it receives data.

 

pg() now looks like this:

 

I also had to change the server so that it adds the header attributes onto the message, like the time and the size.

 
tx()

Much like pg(), we will be reusing the code to verify transactions, so I split it up into a separate function. This way, it can be used across the program and not just in this context.

transaction()

transaction() will take in a message's body, then iterate through the inputs to see if they are valid, using ecdsa.verifyMsg(). The message is the amount plus the recipient's address plus the time. It is a subroutine and does not return anything.

 

It also needs to check that the wallets aren't overspending. To do this, I used a function that will be created later, blockchain.checkBalance(). We simply pass it the amount and the wallet, and it will return true if the wallet has enough arbitrary units to complete the transaction. However, you could bypass this by using the same wallet twice in the same transaction. Therefore, it throws a parse error if a wallet is repeated by adding the wallets to a list and checking each wallet against the list.

 

Now, all tx() need to do is call that subroutine, and then add it to txpool.json, send it on to all the other nodes, and add it to txpool.json, if it's not already there.

 
bk()

As with tx(), I made a separate subroutine to see if a block is valid. This iterates checks the difficulty of a block and then goes through the transactions, calling transaction() for each one. However, I ran into a major problem, in that I could not figure out how to verify the difficulty if it could change. At the moment, the system only lets blocks into blockchain.json if the block is valid. However, with dynamic difficulties, the block could be invalid (by having a difficulty that's too small) but still be in blockchain.json if we don't have it's parent on disk to verify that. This would lead to a lot of issues if the blocks in blockchain.json could not be trusted, and would require an extremely complex restructuring of the whole system, or would require running this subroutine on the whole blockchain every time a block is valid, which would be far too resource-intensive. I could not figure out how to solve this issue, so unfortunatly I had to instead set the difficulty to 6, permanantly. I set the difficulty as a const at the top of the program.

 

The bk() function itself is very simple, as it just calls block() then blockchain.addBlock(), a function that will be covered later.

 
hr()

To construct a bh, we need to use a function that has not yet been covered, blockchain.getTopBlock(). This simply returns the hash of the top block of the blockchain.

 
nr()

For a node request, we simply iterate through connections, and add them to an array if "advertise" is "true", the IP is not the same as the node that sent us this message, and if the number of connections is less than the number that was requested. We also need to make sure that if msg.max does not exist, we use Infinity instead since i will always be less than Infinity.

 
cr()

When a client receives a chain request, there are three possibilites:

  1. It asks for a hash, which it has
  2. It asks for a hash, which it doesn't have
  3. It does not ask for a hash

If it asks for a hash and the client does not have that hash, or if the requested hash is not a part of a complete chain, then it should throw a "notfound" error. If it asks for a specific hash and it has that hash, then the client will call blockchain.getChain() to get the desired chain.

If no hash is requested, then it uses blockchain.mainChain() to get the tallest chain.

 

Sending Messages

We now need to update the sendMsg() function, so that it automatically adds the time, the version number, the hash and the size to the header and body. It also needs to check that the message has not already been sent by checking the hash against sent.json.

I also had an unexplained bug where it would fail to parse sent.json, so line 6-8 detects and fixes that, as I was not able to find a proper fix.

 

I also created a function called sendToAll(), which I have used previously. Unsuprisingly, it reads connections.json and sends a message using sendMsg() to each of the nodes.

 

Parsing replies

Now we need to make the equivalent function to parseMsg() for replies, which is parseReply(). It is nearly identical except it doesn't send a reply back, so is simpler. All it does in the case of error is save the message to error-log.json.

 

Parsing different reply types

Now we need to make the parsing functions for the replies, in parse.js. I have already covered parse.pgreply() in the previous section.

cn()

In theory, we should just be able to iterate through the nodes and add them one by one. However, an oversight in the blockchain.addBlock() function means that it takes in full messages rather than just the body. I worked around this by putting each block into an object as the "body", like so:

 
nd()

When we receive these nodes, we need to make sure that we are not pinging nodes that we are already connected to. This means that we need to get the list of nodes and only send pg messages to those that aren't on both lists.

The first part of this function is simply constructing the pg message.

 
bh()

This is the reply to a hash request, and is the hash of that client's top block. In this function, we need to check if it's in the blockchain, and if not, send out chain requests. To do this, we use !Object.keys(mainchain).includes(msg.body.hash), which gets all the keys from a block and checks to see if msg.body.hash is one of them. If it does, it returns !true, which is false.

 
ok()

I didn't even make an ok() function, I just had it print "Message received ok" into the console back in parseReply().

er()

All we need to do here is append the message to error-logs.json.

 

Exporting functions

Finally, I exported the functions.

 

Blockchain

We now need to tackle the blockchain, which is a critical part of the project. It it basically a cool name for a linked list. We need to create the following functions to process it:

To aid with the first function, I decided to structure the blockchain as an object literal rather than an array, with the hash of the block as the key. In this way, we don't need to store the header, and to get a block we simply use blockchain[hash_of_block].

I creates all these functions in a file called blockchain.js. The blockchain itself is stored in blockchain.json, in %APPDATA%. First of all, I created getBlock().

getBlock()

This simply uses file.get() to get the block

 

Balances

Since we don't want to have to trawl through the blockchain to check every input of every transaction, I decided to store just the balances in a new file called balances.json. This file is again a dictionary-style object, with the wallet as the key storing the amount assigned to them. This is much more efficient for getting the amount assigned to a block. We will only need to generate this file when the blockchain changes. To generate it, I created a function called calcBalances()

calcBalances()

calcBalances() needs to iterate through all the inputs in each transaction in each block in the blockchain. I decided to use forEach() to do this, as although it is technically slower, it is much clearer to see what is happening. In each input, it sees if balances, the object that stores the balances, contains the wallet referred to in the input, using hasOwnProperty() which returns true if the property has a value assigned to it. If that's case, it deducts the amount defined in the input, and increases the recipient's balance by the same amount.

 

However, we also need to accound for the mining reward. I set this as a const at the top of the function. Since it is in microau, it is set to 50000000. For each block, we add the mining reward to the miner's wallet.

 

However, we have a problem. In it's current state, it calculates the balance of every block rather than those under the top block, which is incorrect. What we need is a function which gets the top block then returns a subsection of the blockchain containing only blocks under the top block. This function will be called mainChain(), and will be defined later. For now, we will pretend that it exists (as I made these functions at the same time).

 
getTopBlock()

To get the main chain, we need to know what the top block is. Initally, I iterated through them and based it off of height, using time as a tie-break. To make it more flexible, I made it so you have to pass the blockchain to the function to avoid repeating reading the file.

 

However, someone could submit a phoney block that has a really high height, without actually being connected to the genesis block through the chain. Whilst it is not the most efficient solution, I decided to only consider a block as the best if I could iterate down to the genesis block, which has parent of '0000000000000000000000000000000000000000000000000000000000000000'. I also made it start with the genesis block.

 
mainChain()

mainChain() pretty much repeats what getTopBlock() does to verify that a block is a part of the chain, except it stores the blocks that it finds, to create a subsection of the chain. Only this part of the chain is valid, which is why it is so important.

 
getChain()

getChain() gets a specific part of the chain under a hash passed to it as a parameter, rather than the one given by getTopBlock(). Other than that, it is very similar to mainChain(), other than it has more error handling since it is unknown if the requested chain actually reaches the bottom, unlike in mainChain().

 
checkBalance()

We need a function which checks the balance of a wallet to see if it greater than or equal to some amount. This function is called checkBalance().

 

addBlock()

The addBlock() function is fairly simple, as all it needs to do is check if it's valid, append the block to blockchain.json and then call calcBalances(). However, we also need to remove any transactions from txpool that are in the block. To do this, we iterate through the transactions listed in the block and use splice() and indexOf() to remove the transaction from txpool, if it is there.

We then store txpool.

 

parse.block() is used to make sure that it's valid, and will throw an error if it's not - this is why there is a try...catch statment.

Finally, I exported all the functions.

 

Pages

Before we start creating the pages, I decided to restructure the application slightly. I moved all the Javascript files relating to any of the pages - overview.js, wallets.js etc - to a subfolder in js, pages. I also moved the changePage() function from renderer.js to it's own file, then imported it back into renderer.js. I also modified it to account for the new location of the Javascript files.

 

I imported it back into renderer.js like so:

 

Now that the Javascript files for pages are a folder deeper than the other files, we have to import these other files like so:

 

Transactions

Wallets

The Wallets page will contain a list of all the wallets that the user has stored. The user can interact with each one. The options they will have will be:

First, we need to store the wallets. Each wallet has four attributes:

The first three are static. However, the money in the wallet is dependent on the blockchain, which we do not want to have to trawl through every time to find the value of each wallet. Therefore, I decided to update the wallets every time calcBalances() is called. It iterates through wallets.json, which is where the wallets will be found, and finds the total amount for each wallet. While doing this, I realised that we could take this opportunity to calculate the balance counter in the corner of the application. For each wallet that is iterated through, the amount calculated for each wallets is added to a counter, and the total in the corner is set to this amount.

Finally, the new wallet's values are stored in wallets.json.

 

Finally, we can start displaying the wallets in wallets.html. The HTML structure of the page is like so:

 

The button with an id of create needs to link to the page where wallets are generated, so we will need to import changePage() into wallets.js. #wallet-list is the html object where we will be adding the wallets. .list is a CSS class like so:

 

This means that if the contents of #wallet-list is too wide it will simply be hidden, but if it too tall it will create a scroll bar.

Elements within .list will have class .list-item, which has the following properties:

 

In wallets.js, we first import file and changePage(), and add an event listener to change the page to wallets-create when the button is clicked. This will be the page where we can create a wallet.

 

We now need to populate #wallet-list. This is done by getting the wallets and iterating through them, appending a new div for each wallet, which is done by calling document.createElement() and using appendChild() to place it inside of the #wallet-list element. I also called blockchain.calcBalances() to ensure the wallets are up to date.

 

We now need to add the class .list-item to the child element, and add the data. The first is achieved by calling listItem.classList.add('list-item'). The second part however, using the method above of creating new elements using Javascript and appending them one by one, is cumbersome with the large number of sub-elements we need. Therefore, I decided instead to create a string and use innerHTML, which takes a string instead.

 

Notice how wallet.amount is divided by 1000000 - this is because wallet.amount is in au, and we need to convert it to au.

Now, to see if it works, we need to make a way to create wallets.

Creating Wallets

I created a new page called create-wallets, which was linked to earlier. The HTML for create-wallets is like so:

 

It turns out we can reuse the .list-item class as a kind of highlight box. What wallets-create.js will need to do is populate #public and #private with the appropriate key, then add these and the name to a wallet when #create is clicked.

To do this, we need to use the ecdsa module, as well as file and changePage. This is where we can use ecdsa.createKeys() to create the public and private keys.

 
Testing

We can now test both wallets and wallets-create. First, I navigated to wallets.

Wallets page

As you can see, there's nothing there. So, I clicked on the "create" button, which takes us to wallets-create.

wallets-create page

I entered the name "My Wallet" and pressed "Create", which took me back to the wallets page - except now, it had a wallet called "My Wallet", which means that it worked!

Wallets with a wallet in it

Create Transaction

To create transactions, we need to be able to add in multiple input sources. This is much more complex than a single dropdown, as since the input sources will need to be added through Javascript, they can't (easily) have unique ids.

First of all, I created the HTML for the page, in make.html. #error has class .hidden, which gives it property display: none. When we want to display the error message, we remove this class.

Notice how the #inputs div is empty. This is because we will add the dropdowns with in through Javascript.

 

Now, we need to create make.js. In the init() function, all we do is add event listeners to the buttons. We also import all the necessary functions.

 

Now we need to create the function addInput(), which is called when the #addInput button is pressed. It creates a div with the class .input-group. It then adds a dropdown, a line break, and a number input box to that div, and adds a placeholder value to the dropdown. It then appends .input-group to #inputs.

 

However, we need to put the wallets in the dropdown. For this, I created a new function called populateDropdown(), which opens wallets.json and adds option elements to the dropdown with the wallets' name and balance. It also sets the value of the option to the public key, which is important, as this is what will be returned when we get the value of the dropdown if that option is selected. I also called the function in line 10 of the above snippet.

 

Finally, I called addInput() in the init() function so that the page starts with an input. You can see that this all worked:

make tx

Now, we just need to add the sendTx() function. The reason I had structured the inputs in this way is that I knew that document.getElementsByClassName() gives an array of the elements with that class name. Therefore, we can get all the elements with class name .input-group and iterate through them. We can then use childNodes to get the children of each group, then get their values.

 

However, there are some issues with this - first and foremost, document.getElementsByClassName() does not return an array - it returns a HTMLCollection, which is similar but we can't iterate through it. Luckily, this is easy to solve by turning it into an array using Array.from.

Secondly, there are actually three elements within each group, and therefore child[1] returns the br rather than the input that we want. This is fixed by using child[2] instead.

 

Now we need to get data from wallets.json in order to sign the inputs. However, since wallets.json is an array, we can't easily get the private key with the public key, so I decided instead to put the wallets in a new format so you can get the private key from the secret key.

 

Now we can call convert[public] to get the private key. Next we need to create the signatures etc, which is fairly easy, since we have already created the signMsg() function. However, I put it all in a try-catch statement, and if an error is caught it removes .hidden from #error.

If the message manages to reach the end without errors, it checks the message using parse.transaction, sends the message using sendToAll() and finally appends the message body to both txpool.json and recenttx.json, for mining and for view transaction history, respectively.

 

This can only really be tested when everything else works, so this will be covered in the full testing phase.

View recent

This section is very similar to the wallet viewing page, except it reads recenttx.json instead. The HTML, history.html, looks like this:

 

history.js looks like this:

 

Something of note is the date - since tx.time is a timestamp, we need to turn it into something readable before printing it. For this, we use the Date class. Creating a Date object then using toString() turns it into a human-readable date. Initially, I tried to use toISOString(), which creates this:

 

However, that is not very readable. I soon discovered that I could use `toString() instead:

 

That is much better, as it is clear what time and date that represents.

Again, since we need to be able to create transactions to see them here, and since we need the blockchain to work in order to do that, we will have to test this at the end.

Blockchain

The blockchain pages are a critical aspect of the project, as it is here where we mine the blockchain.

Mining

Mining the blockchain, as mentioned previously, consists of performing hundreds of thousands hash operations to find the one that passes a "difficulty test" - in this case, it passes if the hash begins with a certain number of hashes. Obviously, this is very CPU intensive, and since Node.js is single-threaded this would cripple the performance of the application. This is not desirable, so I looked for alternatives.

Multi-threading alternatives

The first option I looked at was to see if there was a multithreading module default to Node.js. This lead me to child_process. However, although this appeared to be relevent to my problems, it looked far too complex for this project.

Next, I looked to see if there was anything default to Javascript itself. As it turns out, there is a Webworker API which allows you to run a different JS file independantly from the main program, and also communicate between programs.

An example of a Webworker:

 

However, I immediately ran into a problem when I tried to use Node.js functions in the worker.js file.

 

Webworkers can only use plain Javascript, and don't have access to Node.js modules or features. This is a massive problem, as we need to use cryto module to find the hash of the block at a minimum. I therefore had to carry on looking.

The next place I looked was in npm. Since Worker() was exactly what I needed, I looked for Node.js-compatible alternative.

Luckily enough, I found one. tiny-worker replaces Webworker with the same API but now with access to Node.js functions.

 
Mining page

The idea behind the mining page is that we have a pre element acting as a console, and then also have a button to toggle the miner.

 

I also added a button to clear the console.

The console has the following CSS, to make sure that it is the right size, and has a monospace font.

 

Next I created mine.js in /js/pages.

 

init() adds event listeners to #toggle and to #clear. #clear simply sets the content of #console to an empty string. For #toggle, I created an if statement that switches the text content of the button between 'Start' and 'Stop'. This way, we can do one thing when the button says "Start" and a different thing when it says "Stop".

The next stage is to create the Worker. If the button is set to "Start", then it checks to see if the miner exists already. If not, it creates a new instance of Worker and sets it to miner. It sets miner's onmessage function to add any received data to #console. If the button is set to "Stop", then it sets miner to null and then adds "Mining stopped" to the #console. After both, it toggles the text content of the button.

 

Next we need to create mining-script.js, in the /js folder. For the time being, I just made it post "Hello World" back to the main program.

 

Navigating to mine and clicking "Start" gives:

Hello World

This shows that it works!

Mining Script

We now need to flesh out the mining script. Unfortunately, the following code is very messy, as errors generated in the mining script seemed to disappear or be printed in the command line rather than in the Chromium console, and as such took a great deal of trial and error to get working.

Because of this, I will simply put the final code here rather than go through the process of making it.

As mentioned previously, the difficulty is static, as I could not figure out how to verify it if it could change

 

I created a Miner class, which has multiple functions to recreate some of the functions that exist in the main program.

Starting from the top, the first issue I ran into was that this too seemed unable to require modules. As it turns out, I had to put the absolute file path for it to work, which is why I require the hash module like so:

 

__dirname gets the path up to /arbitra-client, and then we concatenate /js/hashing.js to that to get the path to the file we want.

The next issue I ran into was trying to file.js working. I imported it the same way I imported hashing.js, but it kept on throwing an error that was approximately "remote is undefined". This took a great deal of debugging, but I eventually realised that since it was not an Electron renderer process, it did not have access to electron.remote. This was a real issue, as this is required to get the file path for %APPDATA%.

The solution was to get the file path in mine.js, and send it to the mining script using postMessage(). Since network.js also relies on file.js, I realised that in order to send the block it would need to be sent from mine.js. Therefore, in mine.js, I changed the part of the code that initalises the miner to this:

 

In this code, we get the file path that we need, then post a message to the miner. This way, they can get the path without using electron.remote.

When the miner posts a message, it is checked to see if it a string or not. If so, it is printed to #console. If not, it is assumed to be a block and added to the blockchain and sent to all nodes.

First of all, we need to create the constructor of Miner(), which can be seen here. It receives the file path and sets it as a class property. It then creates the block template, and sets all the variables that do not change from block to block. It also sets some properties that are used later to print the hashing rate later on.

An issue I had was similar to the problem I had with the message replies, as the constructor would end with the block still empty. This was because I was misusing callbacks again, and the block would be sent off before the callback had returned. To remedy this issue, instead of using fs.readFile() I used fs.readFileSync(), which as the name suggests returns syncronously. This meant that it had to wait for the data to return, but since this is running on a new thread it does not matter. This is the reason that I had to change getTopBlock() to be syncronous.

 

These following functions are a part of the Miner class.

To generate a random number for the nonce, I decided to put a wrapper around Math.random() so that it generates integers, by multiplying it by a large number. I decided to use Math.random() rather than the cryptographically secure alternative because being fast is more important than being completely unpredictable, as it just needs to be different from other people's guesses.

 

Although mostly unneccessary, I wrapped hash.sha256hex() so that I could feed it objects and it would stringify() it.

 

Since we can't use blockchain.getTopBlock() as it uses file.js, I had to repeat it in the Miner class. I also changed it so that it was syncronous, as explained earlier.

 

Finally, the most important function mine(). When this is called, it will run indefinitely, hashing the block forever.

It first calls our rand() function to get the nonce, and adds the nonce to the block, as well as the current time. It then checks the difficulty by hashing the block and iterating through the hash. If it begins with as many as as is stated in the block, then it passes. If so, it sends a message saying that the hash has been found, along with the nonce and the hash. It then sends the block, which will be sent on from mine.js. It then clears txpool.json, as those messages have now been sent, and increases the height and changes the parent to the hash that was just found.

 

However, I also wanted it to print to the console occasionally to show that it was doing something. After much trial and error, I found the best way to do this was to use this.t1 and this.t2. t2 approximately the current time, and it is updated after every hash. t1 is set in the constructor. Therefore, finding this.t2-this.t1 will give time time in milliseconds since it started mining. I used this trigger code to run every 10 seconds - if this.t2-this.t1 is larger than 10000 milliseconds, then it resets t1 to the current time, and prints the hashing rate as well as the total number of hashes to the #console.

 

To find how fast hashes are being generated, we use this.dhash, which is the number of hashes that have happened since the last interval. Dividing that by this.t2-this.t1 then multiplying by 1000 gives the number of hashes per second. I cropped that to 3 decimal places by using toFixed(3). I found the total time since it started hashing using tt, which is set in the constructor and not changed. t1-tt gives the time in milliseconds since the constructor was called, which I then converted to seconds and rounded.

This is also a good opportunity to see if any more transactions have been added to txpool.json. We read that file, check to see if it's different, and if it is then we change the transactions in the block. The reason I decided to only do this every ten seconds is that doing that for every hash would slow the whole thing down with the reading operations, and would wear out the hard drive.

 
Testing

I tested this by navigating to the mine page and clicking "Start". After 30 seconds:

mining

After 420 seconds, it still had not found a block. This either means that the difficulty test doesn't work, or the difficulty is too high.

420 seconds

I will investigate this later on. However, this does show that the printing function works.

Viewing

Viewing the blockchain is very similar to the history page and wallets. The HTML, view.html, looks like this:

 

view.js looks like this:

 

The only notable differences is that since blockchain.json is not an array, I used a for...in loop which gets the keys of the object.

I also printed the transactions as raw JSON in a pre element. When stringify()ing the JSON data, I gave it the extra parameters of null and 4 which should indent it with 4 spaces.

This needs the blockchain to work, so I will cover it in the Testing phase.

Settings

Network Settings

The things I wanted to be able to do from network settings are:

The first feature was already made for testing.js, so we can just copy that over. The next three change a file when a button is pressed.

As such, network-settings.html looks like this:

 

I added messages confirming that the action has been completed, but are hidden by default. When the associated action is completed, .hidden will be removed confirming it did something.

Next, I created network-settings.js. The outer section looks like this:

 

This sets the span #curr with the current connection target that we have on file.

Manual Ping

To ping a node, all we need to do is get advertise from network-settings.json, use that to form a ping message, then send that to whatever is in the text box using network.sendMsg().

When that is done, it removes .hidden from #pg-save,

 
Target Connections

"Target connections" is a number that indicates how many connections the network should try to attain. If the number of connections is less that this, it will send out node requests until it reaches that number. By default this is 5.

This code simply uses file.store() to store the number in #target to network-settings.json.

 
Advertise

advertise is the variable used when creating error messages. All we need to do is get a value from the dropdown and store it in network-settings.json.

 
Clear connections

This simply wipes connections.json and calls the connect() function. It also sets #connections, the counter in the corner, to 0.

 
Application Settings

Application settings is even simpler, as all I can think to put in it is:

app-settings.html looks like this:

 

The outer Javascript looks like this:

 

There are some extra inports in this file. We import both fs and file.js, because file.js can only store data in %APPDATA%. dialog is a part of Electron, and as such we need to access it through electron.remote.

I retrieved the version number by using require() on package.json, which is where that is stored. Getting the property version from that gives us the application version directly. We then set the version number using:

 

Next we need to save wallets.json. When the button is pressed, it uses file.getAll() to get wallets.json. It then opens a save dialog using dialog.showSaveDialog(). The "filters" are used for the extension dropdown thing in Windows. In the callback we get the file path, and use fs.writeFile() to save wallets.json to that file.

 

This solution used http://mylifeforthecode.com/getting-started-with-standard-dialogs-in-electron/

Next is the "clear cache" button. This sets all the files to their empty values, which is self-explanatory. It then sets #connections to 0 and calls network.connect().

 

Notice that network-settings.json has it's default values set.

Final Touches

These are minor improvements that were not major enough to warrent their own section.

Moving CSS

I moved the CSS to a new folder called /static, as well as the icons. I also downloaded the font-awesome file to this folder. I then changed index.html to this:

 

I also discovered that the defer tag exists, which only loads the Javascript file when the rest of the page is finished, so I moved renderer.js up to the header.

Height counter

In the original concept, I had included several counters beneath the balance total. I had removed all of these except #connections as that was the only one that worked. However, I liked having multiple counters so I decided to add one back in. Since it is easy to get the height of the top block, I decided to display the length of the blockchain.

I changed index.html to this:

 

.fa-chain is a chain icon. I also added .fa-fw to both icons so that they are a fixed width.

This looks like this:

chain length

Then, at the end of blockchain.getTopBlock(), I updated the counter

 

I also set it to 0 in the "clear cache" function in app-settings.

Icon and Splash Screen

I wanted for the application to have an icon, as otherwise the Electron logo is used. I made this in paint in about 30 seconds:

au icons

I stored this in /static.

In main.js, I set the icon.

 

Something that had irritated me through development is that the application displays the HTML before the Javascript is fully loaded, so there is a time when the buttons are unresponsive but there is no indication that that is the case.

However, since #body is blank until the Javascript loads, after which it is replaced by the associated page, I realised that I could set #body to display the icon, and it would disappear when the Javascript loads. This visually indicates that the application is loaded.

I therefore added the icon to index.html

 

I added the following CSS to centralise it:

 

I then started the application to test it.

splash screen

It worked, and was replaced by the overview page after a couple of seconds.

Overview page

Since overview.js doesn't do anything, and since it is loaded every time the application starts, I realised it would be a good place to make sure that all the JSON files exist. It is pretty much the same as the function in app-settings.js, but only sets a file to their empty state if they don't exist.

 

I also realised that if a person tries to mine the blockchain without a wallet, it would break. Therefore, if wallets.json is empty, I generated a new wallet called "My Wallet". This way, everyone starts with a wallet.

 

I also wanted the HTML on the page to be a helpful introduction on what is possible with Arbitra. Therefore, I changed overview.html to this:

 

Which looks like this:

overview page final

npm start script

To save me running .\node_modules\.bin\electron . every time, I added a script to package.json:

 

This means that I can use npm start in the console to start the application.

Testing

In this section, I will attempt to verify that the application and the network work to the standard of the inital objectives.

For some of the tests, I installed Arbitra on a friend's PC, which was running 24/7 anyway for their project. Luckily for me, they had a static IP, so I used that for the backup server.

Test 1 - Application fresh start

Method

  1. Delete the arbitra-client folder in %APPDATA%\Roaming\arbitra-client
  2. Call npm start

Expected Outcomes

  1. Application opens
  2. overview page is automatically opened
  3. A wallet is automatically created

Test Result

Success

Proof

test 1.1

test 1.2

Test 2 - Creating a wallet

Method

  1. Navigate to wallets page
  2. Click on "Create new wallets"
  3. Enter the name "Test Wallet"
  4. Click "Create Wallet"

Expected Outcomes

  1. wallets-create page opens
  2. A wallet is created with the name "Test Wallet"

Test Result

Success

Proof

test 2.1

test 2.2

Test 3 - Pinging a client

Method

  1. Go to the network-settings page
  2. Enter a valid IP
  3. Click "Ping"

Expected Outcomes

  1. A pg message is sent to the correct IP on port 2018
  2. The hash of the message is added to sent.json
  3. The client replies with a pg message
  4. The IP is added to connections.json
  5. The connections counter is incremented

Test Result

Success

Proof

test 3.1

test 3.2

connections.json:

 

Test 4 - Automatic reconnecting

Method

  1. Restart the application
  2. Wait a few seconds

Expected Outcomes

  1. The IP pinged in the last test is pinged again
  2. The IP is added to connections.json
  3. The connections counter is incremented

Test Result

Success

Proof

test 4.1

Test 5 - Connected to backup

Method

  1. Go to app-settings and click "Clear cache"
  2. Restart the application
  3. Wait

Expected Outcomes

  1. After 60 seconds, a ping is sent to the backup server
  2. A ping is sent back and stored etc

Test Result

Success

Proof

test 5.1

Test 6 - Mine a block

Method

  1. Navigate to the mine page
  2. Click "Start"
  3. Wait

Expected Outcomes

  1. "Path received" and "Block form, mining initiated" are printed to the console
  2. Hashing rate is printed every ten seconds
  3. When block is found, it is printed to the console and then sent to connections
  4. Receive ok message
  5. 50au added to wallet
  6. 50au displayed in the top left

Test Result

Success with minor visual bug

Proof

test 6.1

I missed the first block it mined, so here is the second block

test 6.2

test 6.3

test 6.4

Issues

Since the height of a block is zero-indexed, the block length counter in the top left has an off by one error.

Fixes

In blockchain.js, change:

 

to:

 

Test 7 - Make a transaction

Method

  1. Navigate to make page
  2. Enter the public key of "Test Wallet"
  3. Select "My Wallet"
  4. Enter 25
  5. Click "Send"

Expected Outcome

  1. Transaction is created and sent without errors
  2. Transaction is added to history page

Test Result

Success

Proof

test 7.1

test 7.2

test 7.3

txpool.json:

 

Test 8 - Add a transaction to blockchain

Method

  1. Navigate to the mine page
  2. Click "Start"
  3. Wait until a block has been mined

Expected Outcomes

  1. The balance of the wallets should be split 25 - 125
  2. The transaction should appear in the blockchain viewing page

Test Result

Success

Proof

test 8.1

test 8.2

Test 9 - Save wallets.json

Method

  1. Navigate to the app-settings page
  2. Click "Save wallets"

Expected Outcomes

  1. Save dialog appears
  2. wallets.json is saved to the place selected

Test Result

Success

Proof

test 9.1

C:\Users\Mozzi\Documents\wallets.json:

 

Test 10 - Invalid transaction

Method

  1. Navigate to the make page
  2. Enter amount that exceed's the wallet's total amount
  3. Click "Send"

Expected Outcomes

  1. Error message displayed

Test Result

Success

Proof

test 10.1

test 10.2

Evaluation

Initial Objectives

ObjectiveMet?Comment
The user should be able to construct and send a valid transaction.YesThis does work, provided the wallet has the funds required.
The program should be able to automatically parse, validate, and deal with messages.YesThe parseMsg() calls functions with deal with all of the message types.
The user should be able to mine the blockchain.YesThe user can mine the blockchain using the mine page.
Users should be rewarded for mining the blockchain.YesUsers are rewarded 50au per block.
All transactions should be secured through the Elliptic Curve Digital Signature Algorithm.YesAn ECDSA system was implemented and used.
The user should be able to see sent transactions, their wallets, and the blockchain.YesUsers can view sent transactions, their wallets, and the blockchain through the corresponding pages.
The user should be able to change basic settings.YesThere is both an app-settings and network-settings page with multiple options.
The program should connect to other clients automatically, and default to a IP that is running the program.YesPing messages are automatically sent, with a friend's computer running as a backup node.
The program should be able to detect and reject invalid messages.MostlyIt detects obviously incorrect messages, but the system lacks sufficient depth and can easily be tricked.
The blockchain should function as described by the previous section.MostlyThe blockchain is a blockchain, but the difficulty is static.
The user should be able to interact with the program through an easy-to-use UI.YesThe UI is implemented using Electron and is easy-to-use.
The user should be able to save their wallets.YesUsers can save wallets through the app-settings page.

Personal Thoughts

Whilst as a project, Arbitra has been very successful, I do not feel that I have made a cryptocurrency worth using. Even though it fulfilled all of the project goals in some capacity, the system is far too unstable. The point of a cryptocurrency is that it is built on trusting the mathematics and the protocols of which it consists, and whilst the maths may hold up (hopefully) the protocols are buggy and incomplete. My biggest issue was in the networking which I thoroughly underestimated the difficulty of, which meant that it took focus away from other parts of the app, most notable the lack of a dynamic difficulty.

If I were to start over, my biggest focus would be on ensuring that the protocol was fully planned out and tested. There is a surprising lack of information about running peer-to-peer networks using Node.js, and so a lot of the protocol was guesswork based on preconceptions about how things worked. I would also ensure that the mining system was a bit more planned out, as the unsolved bug in the testing phase proved. I would, however, have used the same approach of using Node.js and Electron, as they were powerful and well-suited for the task, for the most part.

I also am disappointed in the lack of focus on functions that deal with the blockchain. Whilst they work, they are very inefficient and I had to brute-force many tasks due to time constraints.

I am very happy with the UI (although it is irritating I couldn't find icons that fit with the Windows aesthetic), and I am very pleased with how the page system worked out. The code is modular and flexible, and I am happy that I did not need any libraries to deal with the UI except Electron. In fact, a personal goal was not to use modules outside of the standard library where possible, and I managed to end up with only four dependencies, including Electron. Whilst this was probably not the best idea if I was trying to make an actual cryptocurrency, it made me deal with a lot of things on a very low level, especially the cryptography, and ultimately made the project more interesting.

If I had more time, I would:

I had attempted to build the project, but however it was far more complex that anticipated and I had to settle with the npm start script due to time constraints.

Stuart Evans, a fellow student, had this feedback:

The cryptocurrency Arbitra is currently in a robust state, but however as there is no market for it, the currency has no value. As with any cryptocurrency, it requires people to use it in order for it to work. However, it has potential and is a solid starting point from which a cryptocurrency could develop.

The UI is pretty and very usable, however it often lacks feedback when performing tasks.

I think this is reasonable criticism. Getting people to use the cryptocurrency is hard given it's rough state at the moment, however this could be solved with a more interactive user interface (for example, showing what state a transaction is in, progress bars et cetera). This would have to be considered if more time was available.

Code

Here is all of the code in the project, organised by file structure. I have excluded /node_modules, README.md, the licence, and git-related files.

app

package.json:

 

main.js:

 

renderer.js:

 

index.html:

 

static

fontawesome.min.js:

Font awesome v4.7.0 is availible to download here:

https://fontawesome.com/v4.7.0/

au-icon.png:

au-icon.png

style.css:

 

pages

app-settings.html:

 

history.html:

 

make.html:

 

mine.html:

 

network-settings.html:

 

wallets.html:

 

wallets-create.html:

 

overview.html:

 

js

parse.js:

 

network.js:

 

mining-script.js:

 

file.js:

 

ecdsa.js:

 

changepage.js:

 

blockchain.js:

 

hashing.js:

 
pages

wallets.js:

 

wallets-create.js:

 

overview.js:

 

network-settings.js:

 

mine.js:

 

make.js:

 

app-settings.js: